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PREFACE 



This document includes the deliverables promised to the National 
Institute of Education by CSE's Management of Instructional Information 
Systems Project. After an appropriate review of the draft and revision, 
this version is being submitted on December 1, 1984. 

The four sections of this document represent the MIIS project's effort 
to conduct research of relevance to the emerging field of educational 
information *)ystems (Section 1), to connect local school personnel with 
both research-based and field-based knowledge (Section 2), and to disse- 
minate such knowledge via conferences and conference presentations 
(Sections 3 and 4). 

We believe that the work of the MIIS project this year usefully 
gathers many of the threads running through the previous efforts of our 
team. 

Over the years, we have been interested in the local uses to which 
test data and evaluation findings have been put. We have regarded the 
school district - with its central office staff, its lay school board, and 
often, its research and evaluation staff - as one of the key management and 
support elements for encouraging principals and teachers to make classroom 
and school -wide use of collected data. We have assumed that the district 
office itself could analyze and disseminate such collected data to better 
track student learning. Such monitoring would then contribute to policy 
and administrative decisions about matters such as budgets, textbook 
selection, staffing, staff development. 
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In earlier project work, we identified three models districts were 
using to link data with decision making. They included an achievement- 
oriented criterion-referenced test model, a school improvement norm- 
referenced test model, and a staff development state assessment test 
model. Each district had developed idiosyncratical ly , in response to the 
unique characteristics of its internal context and its external 
environment. 

We have recognized and continue to regard as very important these 
interactions among the subsystems internal to school districts, as well as 
the continuing Interaction between the schools and their powerful, often 
turbulent social environments. We believe that understanding these organi- 
zational dynamics is a precondition to understanding how schools can be 
improved; and, more particularly, to understanding how data derived from 
testing and evaluation can both describe the present and influence the 
future. 

The increasing availability of school microcomputers has somewhat 
shifted the focus of our work. It is now possible to develop socio- 
technical Information systems which can take a variety of forms: central- 
ized in the district office, decentralized in school sites, distributed in 
both places. These information systems can input many types of data files 
can provide regular or responsive reports, can serve decision making and 
operational needs. The push of technology makes it imperative to weave 
together existing research and practitioner knowledge about testing and 
evaluation, about schools as complex organizations, and about the install a 
tion and maintenance of technical systems. We have confidence that this 
year's work has responded to that imperative. 



SECTION 1 

LESSONS FOR EDUCATORS FROM THE 
MANAGEMENT INFORMATION SYSTEMS LITERATURE 



Overview 
Outline 

Annotated Bibliography 

Glossary 

Bibliography 
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Overview 

LESSONS FOR EDUCATORS FROM THE 
MANAGEMENT INFORMATION SYSTEMS (MIS) LITERATURE 

This section contains the supporting documents Outline, Annotated 
Bibliography, Glossary, and Bibliography for the final version of our 
review of the literature. Lessons for Educators From the Management 
Information Systems Literature. 

This reveiw of the literature is addressed to two audiences of educa- 
tors: administrators who are interested in supporting the development of 
information systems and evaluators in district offices who may be respon- 
sible for that development. It may also interest school board members, 

m 

principals, and technical assistance providers. The review is targetted 
for publication in a journal such as Educational Evaluation and Policy 
Analysis , Administrator's Quarterly , or Educational Leadership . Assembling 
the documents for the literature review has been more time-consuming and 
complex than we anticipated. The management information field, as might be 
expected, is changing rapidly. Earlier research concerns focused on hard- 
ware and system design; later concerns expanded to include software, 
organizational interfaces, and "people problems." New concerns continue to 
emerge. 

Traditional library search methods provided us with a substantial 
background in the field, but they were only a jumping off point. The 
literature in the MIS field is found under a variety of headings, many of 
which are not cross-referenced in library indices; individuals from many 
specializations are working in common or overlapping areas. 
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Our traditional research approach, involving computer searches, 
journal indices, and noted authors' references lists, gave us a basic 
understanding of concepts and issues related to MIS. We are currently 
working a network of contacts with researchers and professionals in the MIS 
field as a way of identifying current "hot topics" and major research 
concerns. It is through this process, for example, that we have discovered 
the research focus that looks at MIS interaction with power and politics. 

We are still working our expanding network list and have recently 
discovered six university centers doing research on management information 
systems.* We are in the process of contacting researcjj^rs at these 
centers as well as other promising contact^**Vefw*^d to us through previous 
phone and personal interviews. 

The materials in this section therefore represent a true progress 
report. Our work is in transition; the content and form of the final 
product is not yet clear. As we continue to add to the annotated 
bibliography, the outline for the paper will undoubtedly change. As we 
continue our work in school districts and consult with our CSE colleagues 
also working in districts and classrooms, we will refine our view of the 
appropriateness of the MIS experience to IIS in education. 



Irvine (Public Policy Research Organization); at MIT (Center for 
Research on Information Systems); at NYU (Center for Research on 
Information Systems); at Florida IntPrnational University, the University 
of Texas, and at the Unverslty of Minnesota. 
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Outline 

LESSONS FOR EDUCATORS FROM THE 
MANAGEMENT INFORMATION SYSTEMS (MIS) LITERATURE 



A. Introduction 

1. Review Purpose 

a. to overview major content areas in MIS literature 

b. to familiarize educators, particularly administrators, 
evaluators, school board members, with issues/areas of concern 
in MIS literature 

c. to suggest findings which might usefully transfer to educational 
settings 

d. to connect the MIS literature with recent developments in 
educational testing and evaluation 

2. Review Methodology 

a. source identification: types of literature, location, search 
strategy 

b. selection criteria: MIS citations; appropriateness to 
educators; availability; readability 

c. content organization 

B. Review of the Literature 

1. Definitions of MIS: range, focus (Mason & Mitroff , 1973; Federico, 
Brun, & McCalla, 1980; Murdock, 1980; Riley, 1981) 

2. History of MIS: research, practice (Markus) 

3. Typology of issues and perspectives 

- technical, economic, behavioral considerations 

- context, design, installation, evaluation characteristics 

- personal, interpersonal, intra-organizational problems 

(see Boland, 1978; Robey & Markus, 1984; Taggert & Tharp, 1977; 
Tricker, 1977; Zani, 1970) 

4. Understanding the Context 

a. approaches: Management Information Requirements (MIR); 

Information Analysis (lA); Situation Analysis (SA) 

(See Cooper & Swanson, 1979; Gorry & Scott-Morton, 1970; Lientz 
Q & Chen, 1980; Mitroff, Kilmann, & Barabba, 1979; Schewe & Weik, 

ERIC 1977.) l(j 
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b. context factors: personal styles, values, roles, resources, 

motivations/interests, organizational 
subsystems, organizational environments 

{Se« Arkoff, 1967; Argyris, 1970; Carter & Silberman, 1980; 
Cerullo, 1980; Dickson & Simmons, 1970; Driver & Mock, 1975; 
Gingras & McLean, 1981; Kling, 1980; Markus & Robey, 1983; 
Markus.) • 

c. tools for use: data analysis, decision analysis, information 

deficiency analysis, functional decomposition, 
environmental scanning, needs assessments, 
checklists, simulations, negotiations, 
interviews 

(See Cooper & Swanson, 1979; Mendelow, 1978; Srin;vasian, 198__;) 

5. Designing the System 

a. uses: deslsion-supported vs. operational; tracking/responding/ 
interacting; reports 

(See Alter, 1976; Boer, 1972; Caldwell, 1975; Hall, 1979; 
Neumann & Hadass, 1980.) 

b. users: number, needs, styles, interests, resistances, types of 
involvement 

(See Markus, 1983; McLean, 1979; Munro, 1978; Sterling, 1976; 
Swanson, 1982.) 

c. roles during design: outside experts, inside experts, users, 

idea champions 

d. approaches to design: top-down vs. bottom-up, positional vs. 

personal, user vs. analyst involvement, 
linear vs. feedback, fixed vs. adaptive, 
limited vs. extended 

e. sequences for design: linear, loopy linear, plug-in, prototype 
(See Bally, Brittan, & Wagner, 1977.) 

f elements in design: hardware, software, resources, skills, 

financial, data bases, data inputting, 
processing, accessing, reporting 

g. costs: financial /psychological /organizational ; start-up/ 
maintenance; training/support 

h. tools for design: data analysis, design analysis, system 

dynamics, syntactical analysis, structural 
analysis, information flow analysis, process 
analysi s 

ERjC J 1 



6. Installing the System 

a. locating operations: centralized, decentralized, distributed, 

off-site/on-site 

(See Danziger, 1979; King, 1982.) 

b. personnel: accountability, training, revised job descriptions 

and evaluations, supports, resistances 

c. monitoring/debugging/feedback procedures 

7. Evaluating the System 

a. criteria: efficiency" effectiveness, user attitudes, satisfac- 

tion, values, usage, decision, performance, work 
environment, organization-environment interaction 

b. tools: MBO; interviews; observations, record keeping, feedback 

sessions, semantic differential, tracking of relevant 
indicators, cost-benefits analysis 

(See Elam, 1979; Keen, 1975; King, 1982; Knutsen & Nolan, 1974; 
Kraemer & Danziger, 1982; Land, 1976; Swanson, 1982.) 

C. Lessons for Educators 

1. What the MIS literature tells us about information systems: 

a. need to understand complexities of information systems as 
socio-technical processes 

b. need to be knowledgeable about alternatives for context 
analysis, design, installation and evaluation: 

new approaches, methods, tools; existing educational 
approachs which can be adapted 

c. need for front-end time for context analysis and design 

d. importance of initially and continually specifying users, uses, 
cycles, formats 

e. importance of "people issues" such as accountability, conflict, 
commitjnent, trust, time, training, security, resistance, 
responsiveness 

f. need for continuous back-end evaluation and feedback 

2. Relevance to ci'^'rent practice in testing and evaluation 

3. Relevance to current understandings of schools as complex 
organizations 
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Annotated Bibliography 

LESSONS FOR EDUCATORS FROM THE 
MANAGEMENT INFORMATION SYSTEMS LITERATURE 

This annotated bibliography is designed to accompany our review called 
Lessons for Educators From the Managment Informations Systems Literature. 
It may, in addition, stand alone as a reference document.* 

The audience for the review and for the annotated bibliography is 
administrators, school board members and educational technical assistance 
providers. The purpose of the literature review is to overview major 
content areas in the MIS literature, to familiarize readers in the field of 
education with issues and concerns in the MIS literature, and to suggest 
findings which might usefully transfer to educational settings. It will 
also connect current thinking in educational evaluation and testing with 
the MIS literature. 

The more limited purpose of the annotated bibliography is to supply a 
ready reference resource for busy educators who want to be able to access 
key articles without hunting them down in management libraries. This 
bibliography, then, is intended to be selective rather than comprehensive. 
It does not deal with technical issues of hardware, software, or systems 
design and installation. Instead, it focuses on the organizational and 
people issues which arise when conceptualizing, developing and implementirvg 

*It should be noted that this is a preliminary document because we con- 
tinue to find relevant literature under many classifications within tlie 
general rubric of management information systems - e.g., office automa- 
tion, socio-technical systems, decision support systems, management, infor- 
mation requirements (MIR), information services management, etc. 

ERJC Is 
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such systems. It does not illustrate the range of settings - banks, 
manufacturing companies, aerospace companies, retailers, local governments, 
etc. - which have had experience with MIS, but rather focuses on the 
empirical or "wisdom" literature which has developed over the past 15 years 
by researchers in graduate schools of management and centers devoted to 
this area of inquiry, and by practitioners working with MIS in the field. 

The search for this annotated bibliography and its more extensive 
non-annotated bibliographic counterpart proceeded with the usual computer 
searches, supplemented by nominations from experts, course reading lists, 
pursuit of oft-cited articles, identification of relevant journals, key 
articles, etc. 

The criterion for including articles and books in this annotated bib- 
liography included: importance in the MIS field, likely appropriateness 
for educational settings, newness of orientation to educators, ease of 
summarizing major points. 

In each annotation we try to indicate the audience to which the 
article is addressed, the use that educators might make of it, the orienta- 
tion or point of view of the article, and the main points or arguments made 
in the article. In a few cases, we indicate that the article should be 
read in its entirety becuase of its content density or its importance. 



9 



I 



BOOKS 



ft 



ERIC 



15 



10 



Clowes, K.W. The Impact of computers on managers . Ann Arbor, MI: 
UMI Research Press, iyb2. 



This book is addressed to managers interested in the resesarch 
on the Impact of computer technology in organizations, par- 
ticularly in the manager's role. Clowes identifies three 
major categories of problems encountered in organizations: 
technical, economic and behavioral. Technical problems relate 
to Integration of hardware and software. Economic problems 
relate to costs of acquiring and maintaining such systems. 
Behavioral problems relate to the human relations considera- 
tions. The behavioral problems are the hardest to identify 
and resolve. 

Clowes maintains that successful implementation of any MIS re- 
quires intensive planning focused on the system's Impact on 
its users. His approach focuses on behavior, not as a sepa- 
rate entity, but as a phenomenon related to characteristics of 
the organization, situation and information system. 

Clowes includes a comprehensive review of management litera- 
ture related to computers in organizations. The review con- 
tains five major areas: Organizational Characteristics, Mana- 
gers' Situational Roles, Computers and Information Systems, 
Computer Impact Perceptions, and Work Activity Orientations. 
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Davis, G.B. Management Information systems; Conceptual foundations, 
structure, and development . New York: McGraw-Hill, 1974. 



A basic text with chapters that include summary, exercises and 
selected references, many charts, tables and diagrams. In 
three main sections, the book discusses conceptual founda- 
tions, structure of an MIS and MIS development and 
management. 
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Federico, P. A., Brun, K.E., A McCalla, D.B. Management inforrnation 

systems and or ganizational b eh avior . New York: Praeger Publishers, 

ratr 



The authors review the literature concerning the Impact of MIS 
on managerial and organizational behavior. They focus on 1) 
the effects of MIS on managerial performance and decision 
making and 2) the Implications of MIS for organizational 
structure and process. 

One of the most comprehensive review of MIS research to date, 
this book Includes an extensive reference list. 
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Kroeber, O.L. Management information systems; A handbook for modprn 
managers . New YorK: rne Free Kress, iwz. ~ 

Kroeber has designed this book to be a practical guide to 
MIS. however, he includes conceptual discussions of systems 
in general and of MIS in particular so that the manager can 
not only use MIS but can understand them as well. 

His introductory chapters include discussions of computer 
hardware and software; data processing (which he distinguishes 
from MIS), and decision making. 

He describes in detail basic MIS activities: report genera- 
tion, inquiry processing and data analysis. 

He a.lso describes the phases of the MIS Life Cycle starting 
with an Information Need and moving on to Planning, 
Development, Implementation, Operation, and Control 
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Lucas, H.C., Jr. Why information systems fail . New York: 
Columbia University Press, ly/b. 



In this basic book for non-technical readers, Lucas views an 
information system within the context of the organization and 
contends that "the major reason most information systems have 
failed is that we have ignored organizational behavior prob- 
lems in the design and operation of computer-based information 
systems," and concentrated too heavily on the technical 
aspects of systems. He notes that the early history of MIS 
reveals that systems were used to support clerical tasks as 
opposed to management decisionmaking. He reviews a number of 
studies which "support the observation that computer systems 
have had a small impact on the decisions made by most members 
of the organizations, especially management." He attributes 
this to problems such as: users not understanding the output; 
duplication overload or inaccuracy of data, unexpected and 
frequent changes in the functioning of the system. 
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Murdock, R.G. MIS, concepts and design . Englewood Cliffs, NO: Prentice 
Hall, 1980. 

A comprehensive textbooK. Each chapter includes a summary, 
questions and problems, and selected references. The book 
covers a wide range of topics, such as Introduction to MIS, 
the Manager's view of MIS, Planning, Design. Implementation, 
Forms and Reports, Problem Solving and Decision Making, Data 
Base Management. 
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Synnott, W.R., A Gruber, W.H. Information resource management; Opportu- 
ni ties and strategies f or the 198us l New York: John Wiley & Sons, 
BbT 



Two chapters in this comprehensive book seem very applicable 
to district concerns: Chapter 5: Effective User Relations: 
The Care and Feeding of Users, and Chapter 6: Top Management: 
Closing the Communications Gap. 

Synnott and Gruber maintain that effective user relations are 
one of the critical keys to success of MIS. They suggest user 
measurement strategies such as: market research, assessment of 
user IM penetration, user satisfaction surveys, and a backlog 
task force. They describe a host of "user involvement strate- 
gies" including" Foot in the Door, Joint Systems Development, 
Perception Management, User Service Contracts, and Customer 
Service Centers. 

Another critical key to success is top management involve- 
ment. Synnott and Gruber describe Critical Success Factors, 
Decision Support Systems, management graphics, and strategies 
for gaining top management support. 
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The following are good sources for collected articles/ readings 
on MIS. 



Davis, G.B., & Everest, G.C. Readings in management information systems. 
New York: McGraw-Hill, Inc., 1976. 

Dock, V.T., Lucksinger, V.P., & Cornette, W.R. (tds.). MIS; A managerial 
perspecti ve . Chicago: Science Research Association, Inc., 197/. 

Riley, M.J. Management Information Systems (2nd ed.). San Francisco, CA: 
Hoi den-Day, 1981. 
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Ackoff, R. Management misinformation systems. Management Science . 1967, 
14(4), B147-156. 



This has become a classic article In the MIS literature and is 
widely quoted and reprinted. The article comes out of the 
author's background in operations research and management 
science. The article attacks several of the (in 1967) pre- 
vailing assumptions about MIS and then describes a decision 
and analysis approach to specifying information requirements. 



Ackoff s list of erroneous assumptions and his corrections: 

Assumption: managers lack relevant information. 

Ackoff: managers suffer from "an over-abundance of 
irrelevant information." A MIS should replace 
information overload with filtered and con- 
densed information. 



Assumption: managers know what information they need. 

Ackoff: managers often don't know how they make deci- 
sions. They therefore "play it safe" and ask 
for "everything." 



Assumption: more Information will lead to Improved deci- 
sion making. 

Ackoff: many managerial problems/decisions do not 
require more information, but instead can 
benefit from decision rules or performance 
feedback . 



Assumption: better inter-organizational communication 
leads to better coordination. 

Ackoff: if organizational units are in conflict with 
one another better communication hurts organi- 
zational performance. 



Assumption: managers need only to use the information sys- 
tem, not understand it. 

Ackoff: managers should be trained "to evaluate and 
hence control it rather than be controlled by 
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Ackoff, 1967 cont. 



Ackoff s model for system design flows from his view of the 
above assumptions. He suggests 1) analysis of the decision 
system; 2) analysis of the Information requirements into a) 
those for which solution models are available, b) those for 
which heuristics can be provided, c) those for which models 
cannot be constructed; 3) reorganize job descriptions so that 
similar type decisions are aggregated to single decision 
makers; 4) design procedures for collecting, storing retriev- 
ing, and treating Information; 5) design system controls. 
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Alter, L. How effective managers use information systems. Harvard 
Business Review. (Nov-Dec 1976), 97-104. 



A typology of decision support systems Is provided with ex- 
amples of each type. The categorization consists of systems 
which: 

1) Retrieve Isolated data items 

2) Aid ad hoc analysis of data files 

3) Produce standard reports 

4) Estimate consequences of proposed decistOTisHmodellingl - 

5) Propose decisions (optimizing) 

6) Make decisions 

Alter's research with 56 systems revealed that successful 
decision support systems Increased the manager's effectiveness 
in the organization by improving interpersonal communication 
(e.g., by using the system as a "tool of persuasion"), facili- 
tating problem solving, fostering individual learning, and 
increasing organizational control. A major finding was the 
problem with cost-justification of decision support systems. 



This abstract can be found in R. Cooper and E.B. Swanson, Management 
info rmation requirements assessment: An annotated bibliography . Los 
Angeles, CA: Center for Information Studies, University of CaTi f orni a 
Los Angeles, 1979. 
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ris, C. Resistance to rational management system?. Innovation . 1970, 
No. 10, 28-35. 



This article is a provocative and fascinating analysis of the 
concept of resistance to MIS. Argyris says that "with regard 
to MIS, there does exist some valid basis for resistance, or 
at least skepticism." Besides the usual reasons that managers 
give their lack of understanding of MIS and their reluc- 
tance to change to a new technology Argyris says managers 
"begin to realize that fundamental changes will be required in 
- theiif^r'seftaV^tyl es-^ -fflana§eiPU4-4hou^ 

He analyzes the changes that could occur when the MIS expert 
tries to make explicit covert policies, practices and norms. 
"As the informal modes become explicit, information becomes 
increasingly under the control of top management." Middle 
managers, for example, may feel hemmed in, feel themselves 
losing their traditional powers, feel that there is less need 
for organizational politics, more need for intellectual and 
conceptual competence. 

Although the author does not suggest solutions for these prob- 
lems, their identification is likely to be extremely relevant 
to some educational settings. 
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Bally, L., Brittan, J., & Wagner, K.H. A prototype approach to Informatic 
system design and development. Information Management 1 1977 , Nov. 
1977, 21-26.* 



"An information system, as implemented, represents a synthesis 
between what the users want, think they want, or state that 
they want, the designers' Appreciation of the users' wants and 
needs, and the constraints of time, cost, human capability and 
technical feasibility." With this thought and the many infor- 
mation system failures of the sixties, the authors propose 

- alterjiatlvfi--systein_ design and development strategies to the _ 

traditional linear strategy. The strategies discussed are: 

1) Linear - One activity follows logically from its prede- 
cessor. Concurrent activities are not allowed and loop- 
ing back implies deficiencies in earlier work. 

2) Loopy Linear - Same as linear except that looping back 
is acceptable. 

3) Plug-in - Follows linear strategy except allows for the 
phasing of the design and implementation of different 
functions. 

4) Prototype - a highly simplified version of the system is 
built and brought Into operation. Experience gained 
from this system is used to revise system requirements 
leading to the implementation of a less simplified ver- 
sion. This cycle is repeated until a satisfactory sys- 
tem Is achieved. 

The authors feel that the prototype -stratesy is very strong 
when dealing with fluid situations and fuzzy requirements. 



*This abstract can be found in R. Cooper and E.B. Swanson, Management 
information requirements assessment; An annotated bibliography , los 
Angeles, CA: Center for Information Studies, University of California - 
Los Angeles, 1979. 
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Boer, G. A decision oriented Information system. Journal of Systems 
. Management, Oct. 1972, 36-39.* 



Recognizing the difficulty In determining what decisions are 
made by managers, the author presents a method to facilitate 
this process. The method Involves the directing of attention 
towards the organization's resources, both tangible (Inven- 
tory, money, etc.) and Intangible (employee skills, customer 
good win, etc.). Once the resources are Identlf-^ed, their 
associated decisions can be determined along with the Informa- 
tion required to support the decision process. 



*Th1s abstract can be found In R. Cooper and E.B. Swanson, Management 
information requirements assessment; An annotated bibliography . Cios 
Angeles, CA: Center for Information Studies, University of Call f orni a 
Los Angeles, 1979. 
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Boland, R.J., Jr. Control, causality and Inforitiatlon requirement . TIMS 
ORSA Conference, May 1, 19/8. 



The author feels that current approaches to the definition of 
information systems are characterized by "inadequate reflec- 
tion on ourselves as systems designers, or on the social pro- 
cess within which our information services are put to use." 
He states that " . . . we have selectively ignored our own,, 
biases in observing the decision making process, and under-^ 
estimated the complexity and dialectical quality of, social 
reality." As such, an attempt is made to provide an alterna- 
tive basis-i'or-ileftTiIng information needs. This alternative 
basis includes the following considerations: 

1) Information requirements should be viewed as temporary. 

2) Information should support the dialectic process of the 
confrontation between man and his reality. 

3) Foci decision analysis upon what currently is/is not 
and what co' ' '^ be. 

4) Consider n due to social interaction (without 
orders) as wtn c: th^ traditional top down control. 

5) Consider a mtu^^ causality in which things are both 
prerequi slates and proO^icts of each other as well as con- 
sidering the tradi J;iona» isequential causality. 



*This abstract can be found in R. Cooper and E.B. Swanson, Management 
information requirements assessment; An annotated bibliograpny . Cos 
Angeles, CA: Center for Information Studies, University of Caii f orn 1 a 
Los Angeles, 1979, 
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Caldwell, J. The effective report crisis. Journal of Systems Management , 
June 1975, 7-12/ 



The author presents guidance for effective management reports 
focusing on report content design. Guidelines include the 
attributes of a good report (e.g., inclusion of many varied 
visual formats, concise, relevant, necessary, exclusion of 
Information not applicable to management action, etc.), attri- 
butes specifically related to management control (e.g., inclu- 
sion of plan figures; actual figures, variance and trend In- 
formation as well as responsibility), and attributes of report 
identification (clear indexing and good titles showing sub- 
ject, classification, and frequency). Examples of good and 
bad reports are provided. 



*This abstract can be found in R. Cooper and E.B. Swanson, Management 
information requirements assessment: An annotated bibliography . Cos 
Angeles, CA: Center for Information Studies, University ot Caii f orni a 
Los Angeles, 1979. 
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Carter, J., 4 Silbennan, F. Establishing a MIS. Journal of Systems 
' *' * Management , January, 1980, 15-21. 

The authors list what they see as major reasons for the fail- 
ure of MIS to live up to expectations: system-user mismatch, 
bottom-up rather than top-down system design; data which are 
too general and too late; lack of clarity about reasonable 
system expectations. 

They define Information as that data which actively informs us 
about the status of something of Interest to us. "Data be- 
comes Information only when viewed as part of a pattern." 

They suggest a six step process for establishing Information 
requirements: determining managers' needs, e.g., regular 
periodic reports or phased reports; monitoring managers' per- 
formance against their plans; forecasted demands for products 
and services, etc.; establishing differences between short and 
long term planning needs; analysis of standard and routine 
problems which can be addressed through a computerized 
decision-making system. 

MIS can be classified as low level systems which supply raw 
data which users must Interpret, Intermediate level systems 
which allow for selective retrieval, or high level systems 
which have computational abilities. The authors say that the 
greatest potential of a management information system is 
reached. when such a system 1s integrated with a management 
science model . 
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Cerullo, M.J. Infonnatlon systems success factors. Journal of Systems 
Management . Dec. 1980, p. 10-19. 

This Is a sffould-be-read nuts and bolts article with many good 
checklists, outlines and lists. Cerullo identified seven cri- 
tical success factors for computer based Information systems 
and analyzed the reason for their Importance. He Infers from 
these working model for a company to use In deslglning, devel- 
oping, and Installing such a system. Much of the Information 
1s based on questionnaire responses by corporations on the 
Fortune 1000 list. 

The seven most critical factors connected with successful use 
are 1) manager/user attitudes; 2) personnel training; 3) oper- 
ating and middle management involvement in planning; 4) tech- 
nical expertise of DP perspWfel; 5) operating and middle 
management Involvement in an^Wis, design and implementation; 
6) user/manager expertise in making their needs known; 7) the 
use of data-base management systems. The least critical fac- 
tors were 1) manager Involvement in post-implementation evalu- 
ation; 2) use of management science or research-bsed tech- 
niques; 3) MIS steering committee composition or organization 
location; 4) top management Involvement in analysis, design 
and implementation phase; 5) use of external consultant in any 
phase. 
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Cooper, R.B., & Swanson, E.B. Management Information requirements assess- 
ment: The state of the art. Data Ba se. (Fall, 1979). 



Densely written for those familiar with MIS terminology and 
literature, this paper provides a useful and comprehensive 
review of the literature related to the assessment of manage- 
ment Information system requirements (MIR). It begins with 
the idea that MIR is a design process active over the life 
cycle of a MIS. The review uses Simon's three design phases: 
the intelligence phase (focusing of the problem area by ident- 
ifying the dissonance between the way things are and the way 
they should (could) be; the design phase (development of in- 
formal or formal theory about the problem area leading in 
clarification and prioritization of action alternatives); the 
choice phase (evaluation of the alternatives resulting in a 
determination of information system requirements). 

After succinctly summarizing the literature in each of the 
phases, the author suggests areas needing further research. 

In the intelligence phase, the literature suggests two types 
of expert roles: active and passive. Active analyst roles 
could invplve marketing postures, purchase motivation, product 
adjustment, communications, post- transaction analysis. A pas- 
sive analyst role suggests that the analyst wait for users to 
express needs. 

Techniques for determining current status include question- 
naires and interviews as well as methods derived fvom transac- 
tional analysis, Jungian characterization. Intellectual under- 
standing, exchange/bribe/puni shment postures. 

Discussion of the design phase can be arranged according to 
six dimensions: top down vs. bottom up, positional vs. per- 
sonal, user Involvement vs. analyst Involvement, linear vs. 
feedback, fixed vs. adapting systems, and limited vs. exten- 
ded systems. 

Seven methods considered in the literature as useful in the 
design phase include: decision analysis, data analysis. 
System Dynamics, syntactical analysis, structured analysis, 
information flow and process analysis. 

The literature relating to the choice phase is mostly concen- 
trated on tools: goal criterion analysis, utility assessment, 
Baysean estimation, tree structuring, cost benefit analysis. 

The paper identifies deficiencies in the methodologies used by 
the MIR analyst and groups them into three areas: Inadequate 
direction for the foundation and evolution of alternative 
methodologies; insufficient components from which to formulate 
alternative methodologies; insufficient components for the 
implementation of the candidate methodologies. 
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Daniel, E.H. Information resource management: An overview for educators . 
Report to NIE No. lR-58, contract No. NiE-4UU-//-ooib. (Available 
from Information Resources Publications, School of Education, Syracuse 
University, Syracuse, NY 13210. Also available as ERIC Document ED 
244 500.) 



This Is one of the few attempts to link an existing literature 
- in this case, the information services literature - with the 
needs of educators. The paper defines the problem, presents 
the background and then provides some "challenges to educators 
In the future." Although incomplete and not thoroughly con- 
ceptualized, it contains good references and bibliography. 
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Danzlger. The "ski 11 -bureaucracy" and 1ntra-organ1zat1onal control: The 
case of the data processing unit. Soci ol ogy of Work and Occupati ons , 
May 1979, 6(2), 204-226. Also available as Reprint No. RP-36 from Los 
Angeles, C^^": Public Policy Research Organization, University of 
California - Irvine, 1979.* 



Does a "skill bureaucracy"— a professionalized service-provid- 
ing organizational unit with a relative monopoly of expertise- 
--operate with minimal effective control by either top mana- 
gers or the clients of its services? This issue is examined 
for local government data processing units. 



> 



^Abstract found in Publication List, Public Policy Research Organization, 
University of California - Ir^^1ne. 
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Dickson, G.W., A Simmons, J.K. The behavioral side of MIS. Business 
Horizons, 1970, 13(4), 59-71. 



This article Is based on the authors' interviews with 17 firms 
which Identified "people problems" as of greater concern to 
them than any topic In the area of operations management. The 
authors' abstract says: 

To enjoy the technical benefits of management Informa- 
tion systems. It Is often necessary to solve the dys- 
functional side effects stemming from behavioral prob- 
lems - In short, people problems. Reactions to the 
Installation of MIS may range from failure to use the 
output to outright sabotage. The authors Identify three 
types of dysfunctional behavior - aggression, projection 
and avoidance - that may appear In four groups - operat- 
ing personnel, operating management, technical staff, 
and tQp management. Only the technical staff - being 
designers and agents of change - shows no dysfunctional 
behavior. Operating management, the group that should 
enjoy most of the system benefits, goes further than any 
other group In Its resistance, and exhibits all three 
forms. The authors suggests ways of minimizing the 
behavioral problems that may follow Introduction of 
MIS. 

Some of the ways they suggest Include establishing proper 
atmosphere (e.g., top management support, open communication, 
trust), participation (of non-experts, users), clarity of sys- 
tem purpose and characteristics (e.g., maximizing Individual 
and organizational goals, minimizing Initial system errors), 
attention to the "human" scope of systems, reexamination of 
manager's tasks, their priorities, and performance evaluation 
criteria. 
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Driver, M.J., 4 Mock, T.J. Human Information processing, decision style 
theory, and accounting Information systems. The Accounting Review , 
July 1975, 490-508.* 



The authors propose and test a theory of decision style based 
upon the degree of solution focus and the amount of informa- 
tion used. Decision makers are divided Into four decision 
styles gategorles: 1) Decisive - using minimal amount of data 
to generate one firm opnion; 2) Flexible - using minimal data 
to derive alternative solutions; 3) Hierarchic - using masses 
of data, carefully analyzed, to arrive at one best conclusion; 
4) Integrative - using mases of data to generate a multitude 
of possible solutions. Proposed attributes to each category 
are: 1) Decisive - concerned with speed, efficiency, and con- 
sistency; 2) Flexible - associated with speed, adaptability, 
and Intuition; 3) Hierarchic - associated with great thorough- 
ness, precision, and perfection; 4) Integrative - experimen- 
tal, creative, information loving style, prior research has 
shown that decision makers may vacillate between decision 
styles; this may happen, for example, when the environmental 
load (i.e., its compleity, threatening or positive aspects) 
changes. One common mixed style (Integrative/hierarchic) is 
labeled complex. An experiment with 54 MBA stdents in a simu- 
lated production and manufacturing environment was performed 
to see if the proposed theory explained differences in the 
volume of information used (purchased) and the speed of deci- 
sions. It was found that: 1) Complex style used the most 
information; 2) Hierarchic style used more information than 
decisive style; 3) Flexible style used more information than 
decisive style; 4) Decisior* speed general conformed to the 
expected; however, the decisive style was found to be slower 
than all other styles; this was hpothesized due to the effects 
of overload conditions. An extensive survey of Human informa- 
tion processing theory literature is also provided. 



*Th1s abstract can be found in R. Cooper and E.B. Swanson, Management 
information requirements assessment: An annotated bibliography . Cos 
Angeles, CA: Center for Information Studies, University of California - 
Los Angeles, 1979. 
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El am, P.E. User-defined information system quality. Journal of Systems 
Management , August 1979, p. 30-33. 

This easy-to-read but not very comprehensive article takes a 
strong position that "quality as perceived and defined by the 
user is (and always should be) the dominant evaluation crite- 
rion in the effective design of information systems." Reasons 
include reduced frustration for user, reduction of later con- 
flicts, user familiarity and understanding of system, minimal 
transfer of power to the data processing department. 

The author says that designers must build descriptive models 
of user needs and design processes, must pilot test the sys- 
tem before it becomes operational, and must regard design as a 
. step function. He provides seven guidelines which s6em to 
come from his experience within an organization and as a lec- 
turer/author in the field of information resource management. 
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Gingras, L. 4 McLean, E.R. Designers and users of Information systems; 
A study In differ ing p rofile s . Information Systems working Paper 

17-62/ 1§S1. ' 



Gingras and McLean report results of a very Interesting study 
showing why "user-oriented" systems often do not meet user 
needs. The study Indicates that designers' images of users 
differ markedly from the users' self Images. Even more signi- 
ficant is the indication that designers' concept of the "ideal 
user" is closely aligned with the designers' own self profile 
and is not closely related to the actual user profile. The 
authors conclude that designers who claim to be "user orien- 
ted" may be unconsciously seeing themselves as the user of 
their designs. 
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Gorry, 6. A., & Scott-Morton, M.S. Management decision systems; A 

framework for managenient information systems , working Paper #458-70, 
Sloan School of Management, Massachusetts Institute of Technology, 
April 1970.* 



In this much-cited article, the authors combine Anthony's 
taxonomy of management activities (strategic planning, 
management control, operational control) with Simon's views on 
the decision process (structured vs. unstructured decisions 
and the intelligence, design, and choice phases) to derive a 
framework for information system requirements. They propose a 
decision analysis approach focusing on and modelling those 
decisions key to the organization. This modelling reveals 
phases of the decision process capable of being structured and 
thus to be Included in the Information system. Implications 
of this approach include: 1) strategic planning and 
management control InforiMtion systems should not be based 
upon (aggregated) operational control data; if this data Is 
found necessary, statistical techniques should be employed; 2) 
the differing requirements of operational control, management 
control, and strategis planning systems imply different 
hardware and software; 3) information system support of 
strategic decision activities should be individualized and 
need not be efficient; 4) information system support of 
operational decision activities should be efficient, have 
ready access to current data, and be easily changed. 



^tis abstract can be found in R. Cooper and E.B. Swanson, Management 
information requirements as sessment: An annotated bibliography . Los 
Angeles, CA: Center foFTrformatlon Studies, University ot California - 
Los Angeles, 1979. 
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Han, T.P. User need analysis. Journal of Systems Management , January 
1979, p. 12-13. 



This how-to-do-lt two-page article provides a useful checklist 
of questions relating to current operating systems both manual 
and computerized, questions relating to constraints In a pro- 
posed system, and questions relating to an analysis of the 
problem being addressed. 
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.Keen. P.G.W. Computer-bssed decision aids: The evaluation problem. 
Sloan Management Review, 1975, 16(3). 



This article is especially interesting for its emphasis on the 
evaluation of qualitative benefits derived from MIS since the 
concept of better information leading to better decisions is 
hard to assess in a traditional cost/benefit analysis. 

Keen stresses linking evaluation to the goal-setting process 
and dealing head on with questions of qualitative Issues. He 
proposes an approach to evaluation that considers the major 
problem for evaluation to be defining what a successful system 
Is. He suggests that Interested parties top management, 
users, designers — need to negotiate a consensus on "suc- 
cess." They also need to decide what the trade-offs are and 
agree on success indicators. 
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King. Organizational cost considerations 1n centralized vs. decentralized 
compu t1 ng operati ons i working Paper version LWP-119J only available, 
[Los Angeles, CA: PlibHc Policy Research Organization, University of 
California - Irvine, 1982] (published in: R. Goldbert & H. Lorin 
(Eds.), The economics of information processing, Vo. 2 . New York: 
John Wiley & Sons. 1982).* 



Raises a challenge to the assumption that decentralization 
will preserve and/or enhance productivity improvements from 
computing. Examines the iippetus behind moves toward decen- 
tralization, reviews the research on cost-related arguments on 
both sides of the cost characteristic of decentralization, and 
constructs a set of hypotheses regarding problematic "cost 
dynamics" of decentralization suggested by the research. 



Abstract found in Publication List, Public Policy Research Organization, 
University of California - Irvine. 
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King, W.R., & Rodriguez, J.I. Evaluating management Information systems. 
MIS Quarterly , 1978, 2(3), 43-51. 



The authors note that most evaluations of MIS are based on 
efficiency rather than effectiveness considerations, are 
developed post hoc rather than conceptualized along with sys- 
tem goals, are likely to be based on unreasonable and unfeas- 
ible expectations* The evaluation model presented here pro- 
poses assessments of attitudes, value perceptions, information 
usage and decision performance at five points In time: before 
the needs assessment, before the MIS design phase, before the 
MIS development phase, before system Implementation and after 
system Impleme tatlon. An extended example and description of 
measures Is provided for each of the assessments. 
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KUng. Social analyses of computing: Theoretical perspectives In recent 
empirical research. In A. Mowshowltz (Ed.), Human choice and 
computers, 2. Vienna, Austria: IFIP Conference Proceedings. (Also 
avamBTe a? Reprint No. RP-102 from Los Angeles, CA: Public Policy 
Research Organization, University of California - Irvine, 1980.)* 



Provides an orienting perspective toward the study of social 
impacts of computing and discusses the Intellectual evolution 
and social organization of scholarly and professional activity 
In which these studies are pursued. 



''.Abstract found In Publication List, Public Policy Research Organization, 
University of California - Irvine. 
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Knutsen, K.E., 4 Nolan, R.L. Assessing computer costs and benefits. 
Journal of Systems Management , Feb. 1974, 28-34. 



The authors state that computer systems are agents of change 
enabling firms and people to operate in new, more efficient 
ways. As such, the traditional capital investment analysis 
techniques (e.g., return on investment) are not adequate bene- 
fit measures for computer systems. Benefit assessment should 
include equipment displacement, direct cost (e.g., people) 
displacement, indirect cost (e.g., inventory level) displace- 
ment, sales increase (through intelligence, etc.), management 
planning and control impact, and organizational character 
changes (e.g., resulting in a "higher order of discipline). 



This abstract can be found in R. Cooper and E.B. Swanson, Management 
information requirement? assessment: An annotated bibliography ^ Cos 
Angeles, CA: Center for Information Studies, University of California - 
Los Angeles, 1979. 
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Kraemer & Danziger. Computers and control in the work environment . 
Working Paper No. WP-17Z. Los Angeles, CA: Public Policy Research 
Organization, University of California - Irvine, 1982. 



Empirical data from more than 1,50 local government employees 
are used to explore tie effects of computing on key control 
issues in the work environment: control over others, control 
by others, time pressure on the job, and overall mastery of 
one's work environment. The findings are clear and often sur- 
prising, indicating that certain roles have characteristic 
patterns of benefits and losses of control. Overall, the 
"information elite" of staff professionals enjoy the greatest 
increases in control due to computing. 



Abstract found in Publication List, Public Policy Research Organization, 
University of California - Irvine. 



49 



44 



Land, F.F. Evaluation of systems goa' s in determining a design strategy 
for a computer based information system. Computer Journal , 1976, 



290-294.* 



The author describes "... a method for identify and evalua- 
ting the goals of the organization and of measuring the con- 
tribution alternative systems designs may make to the achieve- 
ment of the goals." The approach says to: 

1) Identify groups impacted (both negatively and posi- 
tively) by the system 

2) Relate organizational goals to the groups 

3) Decompose goals into "measureable" form (sub goals) 

4) Assign utilities to each subgoal 

5) Evaluate systems in light of their Impact and the 
util ities. 

An automated tool has been developed to aid in the evalua- 
tion. 



This abstract can be found in R. Cooper and E.B. Swanson, Management 
information requirements assessment: An annotated bibl iograpny . los 
Xngeles, CA: Center for Information Studies, University of Cali f orni a 
Los Angeles, 1979. 
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Llentz, B.P., A Chen, M. long range planning for Information services. 
Long Range Planning , Feb. 1980, 1_3, 55-66. 



The authors are addressing managers 1n an attempt to persuade 
them that long range Information services planning Is a good 
thing. They say that Increased diversity In hardware/soft- 
ware. Increased costs and Increased complexity In the need for 
coordination and planning of distributed computing systems and 
the tendency for outmoded systems to maintain their old In- 
efficiencies even with new equipment make It Important to 
engage In three levels of planning: long range, Intermediate 
strategic planning, and Immedlatlon action planning. The 
steps suggested for doing long range Information service plan- 
ning (LRISP) Include: Understand the Environment, Define the 
Objectives, Develop a Strategy, Suggest Project Candidates, 
Specify Expected Performances. 
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Markus, M.L. The new office: More than you bargained for. 
Computerworld OA , 1984. 

Interesting reading; a good news/bad news approach to Office 
Automation (OA). Recommended reading for managers/administra- 
tors: not only do they deal with office staffs who are invol- 
ved, with Implementing automated processes, but they could 
likely draw parallels from the OA experiences to situations in 
which technology is being Introduced in instructional 
settings. 

Markus identifies two conclusions from research on computer- 
based applications: 

"Office systems must be considered on at least two levels: 
their effect on individual and their effect on the collec- 
tions of people as we know as organizations ... [it cannot] 
be assumed that a system that benefits individuals will bene- 
fit their organization and vice versa." 

"Office systems must be considered not just for their impacts 
on what people do when they work, but also for their Impacts 
on how they work (and how they feel about this) and where and 
when they work .... The social aspects of OA are us impor- 
tant as work task impacts." 

4 

Markus describes potential benefit^ or OA sys1;ems (such as 
reduced time to perform tasks) and explains why expected bene- 
fits do not always materialize (automated systems seem to 
create recurring hassles users.) 

In assessing Impacts of office systems, Markus focuses not on 
technology nor on characteristics of people and organiza- 
tions. He approach is to look at "the way particular system 
features interact with a particular organizational setting." 



o 
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us, M.L. Implementation politics: Top management support amd 
user Involvement. Systems, Objectives, Solutions , 1981. 



Markus discusses resistance In terms of the organizational 
power structure. She questions the automatic use of user par- 
ticipation; she maintains that it is not appropriate in all 
cases. 

Her thesis is that "... causes of resistance He in the 
degree to which a system conflicts with the existing power 
structure in the organization." 

"Resistance can occur Independent of user participation and 
top management support. Users may unwittingly participate in 
the creation of an organlzatlonl ly-dissonant system design 
which they later resist when its implications are felt. 
Organizationally appropriate designs are frequently adopted 
willingly regardless of who suggested them or developed the 
specifications. The presence or absence of Implementation 
tactics like user participation cannot produce accepted or 
successful systems in and of themselves, but they may be 
instrumental, in a secondary way, in affecting the degree to 
which a computer-based system matches or diverges from the 
organizational power structure." 
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Markus, M.L. Power, politics and MIS Implementation. CoiTwminl cation of the 
ACM, 1983, 26(6). ^ 



A refreshingly different perspective on resistance. Rased on 
"Interaction theory," this perspective does not view resis- 
tance to MIS as Inherently bad, but views It as a phenomenon 
which may be a signal that "an Information system Is altering 
the balance of power In ways that might cause major organiza- 
tional disfunctions." 

Markus describes three theories that can be used to explain 
why resistance occurs: 

1. Internal Factors — such as people, In general, res'"st 
change. 

2. External Factors — Involves factors Inherent In the appli- 
cation or system being implemented, such as technical 
qual Ity. 

3. Interaction Theory — interaction of the system and the 
context of use cause the resistance: 

The "Sociotechnlcal variant" looks at the Interaction of 
the system with the distribution of intra-organ1zat1onal 
power. 

The "Political variant" looks at the Interaction of the 
system with the distribution of 1ntra-organ1zat1 nal 
power . 

She says, "It should be noted that this explanation Identifies 
neither the system nor the organizational setting as the cause 
of resistance, but their Interaction .... The interaction 
theory can explain different outcomes for the same system in 
different settings." 

"The interaction theory has the apparent disadvantage of pro- 
viding no universal, noncontingent advice to systems analysts 
and management implementors of systems. But it is more useful 
than other theories for predicting resistance and for genera- 
ting varied and creative strategies that will help both to 
prevent it and to deal with it when it arises." 

Markus recommends implementors use self-examination strategies 
to understand other people's reactions. She also warns that 
"the analyst should recognize that the goal of the exercise is 
not to 'overcome' resistance, but to avoid it, if possible, 
and to confront it constructively, if not . . .. Resistance 
is not a problem to e solved so that a system can be installed 
as intended; it is a useful clue to what went wrong and how 
the situation can be righted." 
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Markus, M.L., & Robey, D. The organizational validity of management 
Information systems. Human Relations, 1983, 36(3), 203-226. 



The authors say that accumulating research indicates that 
while the technical attributes of a system may be necessary 
for system success, at least in some threshold quantities they 
are not sufficient for system success. 

They provide a conceptual framework for understanding the 
organizational validity of MIS systems. The prevailing view 
Is that an organizationally valued system is one in which key 
attributes of the system match users' psychological character- 
istics. "In our conception, the fit between the system and 
users' motivations or cognitive styles is only one of four 
ways In which a system can match its context of use. The 
others include the structiwal dimensions of ' an organization, 
distribution of power in the organization, and the Interface 
between the organization between the organization and the 
environment." Then they go on to say "a second key aspect of 
our conceptualization Is that organizational validity Is a 
property neither of the system Itself, nor of the organization 
In which it 1s used, but rather of the degree of fit or match 
between them." Finally, they say, "organizational validity is 
a useful concept" but- recommend "caution in applying it norma- 
tively." That is, "an organizationally valid information sys- 
tem might be easily Installed but fail to pro<^'4ce any signifi- 
cant benefit because It merely automates inefficient organiza- 
tional rules of thumb." 
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A fho ;«rademic research community 
This .rtUle Is '■l''':"!!^,,!'' ^^Jrc^^^^^i^ on MIS. The 
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tlon System (MIS), it ^ s ^,5^ only one under- 

systematic resear.h program on HIS. 



5*, 



51 



Mason, R.O., & Swanson, E.B. Measurement for management decision: A 
perspective. California Management Review , 1979, 21_{3), 70-81. 



This article Is Intended by the authors to be a contribution 
to that management literature which 1s concerned with develop- 
ing a general theory of measurement in the managerial 
context. 

The thrust of this article Is that "measurement Is a fundamen- 
tal process of management." Since the manager is the user of 
the measures, the system for measurement must take into ac- 
count the managers' disposition and intentions rather than 
traditional scientific measurement techniques. Three kinds of 
managerial dispositions are identified by the authors: "What 
problem shall I look into?" (attention directing); "What 
course of action is better?" (problem solving); and "How well 
am I doing?" (scorecard keeping). This means that the de- 
signer of a measurement system must have intentions consistent 
with the client's values and must design a system that guaran- 
tees that their mutual and compatible intentions are 
realized. 

In the authors' terms, "the actual implementation of a mea- 
surement system takes the form of a management information 
system." The primary functions of such a management informa- 
tion system are data gathering, data processing, and manage- 
rial Inquiry and deciding." 
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McLean, E.R. End users as application developers. Information Systems 
Working Paper , Center for Information Studies, Graduate School of 
Management, UCLA, 1979, 8-79. 



This paper emphasizes the ever-changing needs of users and 
addresses issues of design, prototyping, and maintenance. 

McLean proposes having end users function as their own system 
developers (as opposed to DP personnel) so they can create and 
modify their own applications as needed. For this purpose, he 
recommends establishing a use-friendly environment which takes 
into account user frustrations and terrors and their need to 
change the system as they use it. 
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Mendel ow, A.L. Environmental scannning - The Impact of the stakeholder 
concept. Proceedings of the International Conference on Information 
Systems > 1978, pp. 407-417. 



The open systems perspective emphasizes the importance of 
environmental considerations in strategic planning. This 
article mentions four modes of scanning (undirected, condi- 
tioned, informal search, formal search), six foci for the 
scanner (the image, the customer, the potential customer, the 
competition, the regulators, and the critical intelligence 
providers), and three schedules for scanning (crisis-oriented, 
periodic, continuous). 

The author accurately notes that these frameworks do not tell 
a manager how to do environmental scanning. Me proposes a 
stakeholder framework (referenging Mitroff and Mason, 1980; 
Ackoff, 1970; Ross & Goodfellow, 1980) that categorizes stake- 
holders as shareholders, government, customers, suppliers, 
lenders, employees, society, competitors. He notes that since 
the purpose of environmental scanning is to reduce uncer- 
tainty, stakeholders may be categorized into one of four qua- 
drants using Power (Hi-low) and Dynamism (Hi-low) as axes. 
Each quadrant has its own scanning schedule and process. 
Stakeholders are assigned to quadrants. 

This is an Interesting conceptual article which suggests a 
start on a practical how- to-do-it technique. However, it does 
not go far enough to be useful to a manager. 
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Mitroff, I.I, Kllmann, R.H., & Barabba, V.P. In G. Zal tman (Ed.) . 
M anagement principles for nonprofit agenc e s and organiz ations. 
Vo'rk: American Management Associations, 1979. 



This Is a key must-read article In the MIS literature. It 
builds on Ackoff's (1967) classic article Identifying an array 
of erroneous assumptions and adds five other areas. 

1. Solving the right problem. This means the design must ask: 

What are the classes of problems to which the system will 
restrict Itself? Are they well known beforehand or unspe- 
cificed? Fixed or changing? 

2. Involving the right people in the design. "It is vitally 
Important to get as many potential users, clients and 
stakeholders of the system deeply involved in as many 
phases of the MIS design as possible." 

3. Different kinds of evidence. The Issues include evidence 
for Whom? Evidence for what? 

4. The System's Boundaries. "Many MIS designers implicitly 
take the concept of an information system to be synonymous 
with that of a computerized system . . . Such a limitation 
need not always be the case ..." 

5. The Decision-Information Structure. Using the Montgomery 
and Urban (19 ) model of an MIS as composed of four banks 
- a statistical model bank, a display system, a model bank, 
and a data bank - the authors add a problem bank, a strate- 
gic assumption bank, and a decision maker bank. 

The authors describe eleven principles for avoiding a mis- 
information system, and outline in great detail a stakeholder- 
based process for a MIS. 
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Munro, M.C. Determining the manager's infcjrmatlon needs. Journal of 
Systems Management , June 1978, 34-39. 

The author compares and contrasts the data aniysis and deci- 
sion analysis approaches to Identifying management Information 
needs. Data analysis advantages are stated as 1) cost-effect- 
ive for structured decisions; 2) may provide more flexible 
information flow. Data analysis disadvantaged are: 1) re- 
quires managers to articulate Information needs; 2) informa- 
tion is not linked to organization's objectives; 3) there are 
no established procedures or standards. Decision Analysis 
advantages are stated as: 1) explicit linkage between infor- 
mation and organizational objectives; 2) provides Inform re- 
sults independent of the analyst Involved; 3) good for un- 
structured decisions; 4) information is tailored to the per- 
sonal decision style of manager; 5) Improves decisions as well 
as information. Decision Analysis disadvantages are: 1) in- 
formation requirements may change when the manager is 
replaced; 2) decisions are difficult to model. 



'This abstract can be found in R. Cooper and E.B. Swanson, Management 
information requirements asse« ;<;ment: An annotated b^bl 09^Y; Jf l 
Angeles, Ca: "Center for In formation hud\es. University of call f orn 1 < 
Los Angeles, 1979. 
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umann, S., 4 Hadass, M. DSS and strategic decisions. California 
Management Review , 1980, 22(2), 77-84. 



An easy to read Hscusslon of the Interaction betwen computer- 
ized Information systems and strategic decision making. The 
authors, based on their literature review, assert that the 
major reasons for the lack of sufficient impact of information 
systems on such top level decision making is due to 1) press 
to get such systems operating quickly. This is easier to do 
at the operational control level which Is easier to do than at 
the strategic planning and management control level; 2) MIS 
contribution to top level decision making is regarded as a 
by-product of the routinely generated operational information 
rather than as an Independent activity; 3) the undefined 
nature of both the decision-making structure and the products 
(i.e., reports) it needs from a MIS. 

The article classifies decisions into structured, unstructured 
and partially structured. "The process of making a completely 
structured decision is algorithmic (logical, quantitative, 
unequivocal, entirely defined). All of the alternatives and 
the consequences of their implementation are known and quanti- 
tatively defined ... The process of making an unstructured 
decision is heuristic ... We must resort to hypotheses, 
intuition, evaluations, educational processes, experience and 
such. It is a decision mode under uncertainty that the alter- 
native selected is optimal, so there is no predefined or best 
approach to making such a decision" (p. 78-79). 

The authors then argue that a MIS can have two logically dis- 
tinct components, one to support structured decisiofi making, 
the other to support unstructured and sem1 -structured decision 
making. They suggest Independent information systems which 
are separate from one another in terms of aims, structure, 
location, staffing, status, methods and resources but which 
are "confederated" to take advantage of one another's 
existence. 
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Robey, D., 4 Markus, M.C. Rituals In Information system design. 
MIS Quarterly, 1984. 



The authors point out that political factors have been largely 
Ignored In IS research with the exception of Argyrls (1971) 
and Mason and Mitroff (1973) who "mentioned the relationships 
between Information and power In their seminal article on IS 
research." 

The system development life cycle goes from project Inception 
to a feasibility study, systems analysis, systems design, spe- 
cification perspective, programming, testing, training, in- 
stalling, and operating. 

The authors see that political persepctives underlying the 
seemingly rational motivations for systems design. They note 
that a political perspective would not imply that user in- 
volvement techniques such as steering committees, informa- 
tional analysis requirements, prototyping, or behavioral 
approaches, are inappropriate. Rather, that there may be 
underlying differences in the motivations of various actors 
and the opportunities which system development offers them may 
be either to their advantage or disadvantage. 
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Schewe, CO., i Weik, J.L. liulde to MIS user satisfaction. Journal of 
Systems Management . June 1577, 6-10. 

The authors propose a marketing approach to the design and im- 
plementation of management Information sytems. This approach 
Includes: 1) Market delineation - determine potential users 
and their characteristics; 2) Purchase motivation - assessment 
of factors Influencing purchasing behavior; 3) Product adjust- 
ment - matching product to market; 4) Physical distribution - 
movement of Information products; 5) Communications - creation 
of favorable selling climate through communication with poten- 
tial/current users; 6) Transaction - the encouraging of system 
usage after purchase; 7) Post- transaction - feedback for 
effective mari.*et1ng. 



This abstract can be found in R. Cooper and E.B. Swanson, Management 
information requirements assessment: An annotated bibl iograpny . los 
Angeles, CA: Center for Information Studies, University of California - 
Los Angeles, 1979. 
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Srinvasan, A., 4 Kaiser, K.M. Information deficiency: Implication for 
Information systems design. Proceedings of the International 
Co nference on Information Sys tems, 1^8 , 77-88. 




This article Is not of much direct use to educators. However, 
It Introduces an Interesting notion - that of Information 
Deficiency - as a way of focusing on users' Information 
needs. The gap between the need strength and the user-per- 
ceived degree of availability of a particular Information 
category Is termed Information Deficiency (ID). The authors 
say: 

The factor determines how critical a particular Informa- 
tion category Is therefore not only dependent upon how 
much It 1s perceived as being needed by the decision 
maker, but also on the factor of how difficult the user 
perceives It Is to obtain Information of that particular 
category. The Implication here Is that, a patlcular 
information category may not even be a part of ';he eli- 
cited needs, simply because the prevailing perception of 
the user may be that the changes of obtaining It are 
very low . . ." 
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Sterling, T.D. Humanizing computerized Information systems. The Journal 
of the Association for Educational Data Systems. Fall 1976, iO.* 



The author points out that "... design strategies that 
account In large part for the presence of dehumanizing fea- 
tures In a management system . . . [Include treating] . . . 
the recipients of the service and participants In the systems 
as unpaid components whose time, effort and Intelligence do 
not appear In the cost accounting." He thus presents twenty- 
five guidelines for humanizing Information systems. The areas 
dealt with by the guidelines are: 1) procedures for dealing 
with users; 2) procedures for dealing with expectations; 3) 
system action with respect to Information; 4) the problem of 
privacy; 5) the ethics of systems design. 



This abstract can be found In R. Cooper and E.B. Swanson, Management 
Information requirements assessment; An annotated bibliography , los 
Angeles, CA: Center for Information Studies, University of California - 
Los Angeles, 1979. 
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son, E. B. Measuring user attitudes In MIS research: A review. 
Omega. The Institute Journal of Managment Science , 1982, 1£(2), 
157-165. 



This article attempts to define user attitudes toward MIS and 
seems to Indicate that we don't know all the components of 
user attitudes toward MIS but that they are probably Influ- 
enced by cognitive set, attitude toward people running the 
MIS; disposition to MIS kinds of Information; experience with 
previous reports generated by MIS and previous experience with 
the process of using MIS. 

The purpose of the paper Is to review user attitude treasure- 
ment In MIS research. The motivation for doing user attitude 
research In MIS Is two-fold. First, user attitudes are 
assessed for the purpose of contributing to a theory of MIS 
development and explaining their success and failures. This 
1s called the Implementation perspective. A second purpose is 
to understand how users are informed by an information sys- 
tem. User attitudes are seen from this perspective as a de- 
pendent variable in studying the value of information sys- 
tems. Attitudes, for the purpose of the paper, are defined as 
"a learned predisposition to respond in a consistently favor- 
able or unfavorable manner with respect to a given object." 

The author suggests that a concept of channel disposition 
might be useful for further research. Channel disposition is 
conceived as evaluations about both the quality of the infor- 
mation and the quality of access. Quality of infomiation 
refers to the value placed on the outputs of the information 
system; quality of access refers to the value placed on the 
process of obtaining the outputs. Thus, channel disposition 
is an Indicator of the net utility of the information system 
to the user. However, it does not intended to represent the 
whole of an individual's attitude toward MIS. 
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Taggart, W.M., Jr., & Tharp, M.O. A survey of Information requirements 
analysis techniques. Computing Surveys . 1977, £(4), 273-290. 



This article reviews a variety of approaches to the determina- 
tion of Information needs which has been Identified as "a ir,ost 
critical factor" In succesful MIS Implementation. The review 
organizes the management Information requirements analysis 
process into aspects relating to 1) development, 2) reforma- 
tion (characteristics, scope, degree of sophistication), 3) 
decision-making (process, hierarchy, decision-maker), 4) 
organization (environment, subsystems, management function and 
1 evel ) . 

For each of these aspects an annotated bibliography Is pro- 
vided as an appendix. Although the references are by now 
somewhat dated, they provide an Interesting range of experi- 
ence In libraries, agricultural settings, energy agencies, and 
businesses. 
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Trlcker, R.I. The Impact of Information systems on organizational 
thinking. IFIP '77. 213-221.* 



The author explores the "Interrelatedness" of information sys- 
tems and the organzation. He finds that organizational struc- 
ture and management style affect the tyupe of information sys- 
tem necessary and that, conversely, the information sytem 
available affects the crganizai tonal structure and management 
style. Information is defined as the process of deriving 
"surprise" value from data. This process is the result of the 
data, the use, and the organizational /environmental context. 
It is suggested that high value information (i.e., that 
derived from the creative process) is the result oi the inter- 
action between habitually incompatible frames of reference; 
this process can be aided by semi-confusing, Incomplete, and 
conflicting data. Also Included in his discussion are a list 
of Important strategic issues facing management, the attri- 
butes of decisions, and a framework for analyzing informatioii 
systems. 



This abstract can be found in R. Cooper and E.B. Swanson, Managetnent 
information requirements assessment; An annotated bibliography . Los 
Angeles, CA: Center for Information Studies, University of California 
Los Angeles, 1979. 
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Zani, W.M. Blueprint for MIS. Harvard Business Review, 1970, 48(6). 



Zani was one of the first to identify the disappointing re- 
sults from MIS and to trace this disappointment to the early 
bottom-up data-driven approach to MIS development. He argues 
for a top-down approach to MIS design that focuses on deci- 
sions to be made within the organization and on providing 
managers with the Information needed to make those decisions. 



Zani offers his approach as Sn ideal, not a recipe." It is an 
orientation to planning MIS rather than a procedure. He sug- 
gests that different organ'zatlons will follow different pro- 
cedures to accomplish the top-down approach. 

He does offer a series of questions that can be asked to help 
Isolate specific information requirements to support 
decisions. 
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JOURNALS 
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MIS Quarterly 



A quarterly journal. Publis.ed jointly by The Society for 
Managment Information Systems and The Management Information 
Systems Research Center, Graduate School of Business Adminis- 
tration, University of Minnesota. 

This is one journal targeted toward both -he researcher and 
practitioner. Most issues of the journal include articles 
related to "application"; another group is related to "theory 
and research." 
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Harvard Business Review 



Published bimonthly by the Graduate School of Business 
Administration, Harvard University. 

It is targeted toward professional managers with articles from 
academics and professionals. It covers a wide range of timely 
management topics, including MIS, by well-known authors. At 
the beginning of each issue is a section with summaries of 
articles. 
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Management Science 



Published monthly by the Institute of Managment Sciences, 

Issues of this journal tend to alternate focus: one month on 
theory, the next month on application. The articles are pri- 
marily research-oriented and have abstracts and extensive 
reference lists. 
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Sloan Management Review 



Published threp times each academic year by the Alfred P. 
Sloan School of Management, Massachusetts Institute of 
Technology, 

This journal 1s oriented to top-level practitioners. It 
covers a broad range of management topics and frequently fea- 
tures articles by well-known authors in the MIS field. 
Articles are prefaced by an introduction from the editor. 
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Communications of the ACM 

Published monthly by the Association for Computing Machinery. 



Articles tend to be oriented toward technical questions rather 
than management. The journal appears to be very highly re- 
garded by researchers and Includes abstracts and reference 
lists accompanying articles. 
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Glossary '' 

LESSONS FOR EDUCATORS FROM THE 
MANAGEMENT INFORMATION LITERATURE 



Analog iiodel A physical model that acts but does not necessarily look like 
the real -world object 1t represents. 

ASCII American Standard Code for Information Interchange. 

Botton-up An approach to IS design that starts with data that is 

already in the IS or Is readily available for storage in the 
IS. This is contrasted with a top-down or decision-driven 
approach » 

Central processing unit (CPU) The hardware component of a computer, 

consisting of the controller, the arithmetic and logic unit, 
and internal storage, that executes programs and manipulates 
data. 



♦Choice 



In the decision-making process: the selection of the best 
alternative solution to a problem. 



♦Certainty A decision-making environment in which the outcomes of 
future events are known. 



Code 



CPU 



The programming language into which information is 
transformed in order to be processed by the computer. 

Central Processing Unit 



Critical path The sequence of activities, as shown on a PERT or Critical 

Path Method (CPM) network, along which any delay will cause a 
delay in the completion of the project. 



CRT 



Cathode ray tube. An electronic tube (just like a television 



picture tube) used to display computer output, 
'monitor." 



Al so cal led 



Critical Success Factors (CSF) "the few key areas whero 'things must go 
right' for the business to flourish" Rockert, 19 ). 



ERIC 



"•"Most of the terms in this glossary are in common parlance and are 
composites or are paraphrased from text. Where there is wide variation 
among definitions in the field, we have usually included a citation. 

*These entries were excerpted from the "Glossary of Terms" In Kroeber, 
Management Information Systems: A Handbook for Modern Managers , 1982. 

77 



72 



Data In an MIS or data processing context: unprocessed 

Information; the input to an information-processing system. 

Data analysis The use of a terminal or other on-line input/output device 
to perform mathematical or statistical analyses of data 
stored in a data base or master file. 

Data Analysis Approach A way of determining information requirements by 
focusing on the flow of information in the organization. 

Data base Collection of computer-based and non-computer-based files and 
records supporting the information system. 

Data base nanagenent systea (DBMS) A software or software/hardware combi- 
nation that maintains data in direct access storage devices 
and makes them available to application programs or manage- 
ment queries. Data Base Management Systems enable managers 
and other non-programming users to- work directly with the 
data base system, DBMS software packages allow non-program- 
ming users selective access to files and aid in the presenta- 
tion of reports and In the gathering of statistics. No know- 
ledge of computer programming Is necessary to use a DBMS. 

Data elewient The smallest unit of data that can stand alone and convey 
information. 

Data file A collection of related data records. 

Data gat^ierlng A function which "converts primary sensations of the real 
world into data" (Mason & Swanson, 1979). Includes processes 
such as sensing, observing, rendering. 

Data processing (DP) The manipulation of data by a computer to support the 
recordkeeping and report generation activities in an organi- 
zation, 

*Data recofd A collection of related data elements. 

Debugging Finding and correcting errors in computer programs. During 
the implementation phase, getting rid of technical errors and 
problems. 

Dei.1s1on Support Systems (DOS) Refers to information systems designed to 
give support to unstructured and semi -structured decisions 
made by top management in connection with their need to do 
strategic planning or management control. 

♦Decision tre« A branching diagram of the payoffs and probabilities in a 
decision situation. Decision trees are particularly useful 
to analyze multistage decisions. 
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*0etat1 rei Reports that Include Information on all transactions 
within the subject matter covered by the report. 

♦Olstrlhuted systea A system of electronically linked computers. A star 
system uses a large, central computer while a ring s^'item 
consists only of distributed computers. 

*O0S Disk Operating System. Made up of programs that keep track 

of files, save and retrieve them, and do other organizing 
tasks. 

*0P Data Processing 

End User Person who uses the computer program to perform job 
functions, as opposed to the designer, programmer. 

Interiii reports Reports that give only information that falls within 
certain management-defined parameters. 

Exception Reports Reports that list only information that falls outside 
of certain management-defined parameters. 

Feasibility stu<(y A study to determine if a major endeavor, such as 
developing a new MIS, 1s economically, technically, and 
behavioral ly feasible. 

Feedback Output of a system that is used to keep the system under 
control . 

Hardware The physical components of a computer, such as input devices, 
the central processing unit (CPU), and output devices. 

Heuristic prograMrIng The simulation of human judgment within a computer 
program. 

^iMKdlate conversion A technique in which the new system replaces the 

old system in one sweeping change without phasing or parallel 
operations. 

Infornatlon analyst A systems analyst who helps functional usors identify 
information needs and use the MI 3 to satisfy those needs. 

Information Services **lann1ng A name sometimes given to the process of 
understanding the environment and developing an action plan 
to get the system designed. 

Infon«at1on Overload Often used to refer to situations in which MIS 1s 

used to provide managers with more information than they can 
make sense of. 

Interactive Systems User can program the system as well as use existing 
programs. 
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Life Cycle 



Interface The contact point between the Information system and the 
user; If the Interface 1s two-way, the user both receives 
output and furnishes Input. 

^Inquiry processing The use of a terminal or ether on-line input/output 
device to obtain limited information from a data base or 
master file. / 

I/O Input/output. 

IS Information System 

(Also MIS Life Cycle) stages in the development of a MIS. 
Stages can vary but often include: information need; plan- 
ning; development; implementation; operation; control. 

Mainframe A large computer capable of handling many peripherals and/or 
satellite computers. 

Manageiient audit A postinstall ation check to determine whether or not an 
MIS is satisfying the information needs of manaoegment. 

Management Information System (MIS) Definitions vary within the field. 

Refers to an organized set of processes that provides infor- 
mation to managers at all levels to support the operations 
and decision making within an organization ( ). 

Microcomputer A very small (hand-held or desk-top) computer. Sometimes 
called a "personal computer." 

Minicomputer A small -to-intermediate sized computer, often with capab-fli- 
ties that exceed those of all but the largest computers of 
ten years ago. 

Model An abstraction of reality used when the real -world situation 

represented is too complex, too costly, or too time-consuming 
for experimentation. 

Modem An acronym for mo^dulator-demodulator, a data communications 

device that converts digilaT data into an analog, such as a 
modulated sound signal, and back again. Commonly used for 
computers to "talk" to one another over telephone lines. 

Multl -processing A timesharing technique in which two or more programs are 
executed simultaneously. Multiprocessing is a hardware 
feature. 
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''^Itl programing A timesharing technique in which programs are executc^d 
while the CPU is idle with respect to other programs. 
Multiprogramming is achieved through systems software. 

OA Office Automation 

Paperless Office where records and information are kept and transmitted 
electronically 

^Parallel conversion A technique in which the old or manual system 
continues to operate for a few cycles as a check on the 
accuracy of the new system. 

Performance evaluatl^^n and review technique (PERT) A network analysis 

technique used in project management. PERT is particularly 
helpfuT in complex, long-range projects such as MIS 
development. 

♦Perforwance monitor A means, either using software or hardware, of 
measuring the efficiency of a computer in processing the 
application programs of an MIS. 

*Phase-1n conversion A technique in which one program or application at a 
time is introduced until the old or manual system is 
eventually replaced with the new system. 

*P11ot conversion A technique in which the new system is implemented in a 
limited fashion - in one plant or in one product line - until 
it can be determined that the system works and can be 
implemented organization-wide. 

^rograanable decision A decision that can be reached by following certain 
niles that lead to unambiguous results; routine decisions 
involving quantitative inputs and computational processes. 
Also called a Structured Decision. 

*f^uery language (QL) The data base management system language employed by 
users at terminals to make inquiries or perform data anlayses 
on data in the data base. 

*RAH Random Access Memory Stores programs, data, etc., while 

computer is being used. When you turn off the computer, you 
lose what is in RAM unless it has been saved. 

Real-tine processing The updating of a masster file or data base with 
transaction data in time for feedback to influence the 
outcome of the transaction; extremely rapid transactional 
processing. 

Reports 'ine kind of system output which can be performance, progress, 

iture-oriented, requested, routine; exception; detailed; 
.iiterim. 
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Requirements Analysis this can be viewed either as a decision process 

active over the entire life cycle of a management ir formation 
system (Cooper & Swann, 1949), or as a set of techniques 
which gather information prior to one MIS design phase 

Resistance Reluctance of users to use the information system 

*R1sk A decision-making environment in which the outcomes of future 

events are not known, but pru')abil ities can be assigned to 
those outcomes. 

ROM Read Only Memory Built into the computer. It contains all 

the information the CPU needs to get started. Often 
computers come with BASIC stored in ROM. ROM stays in the 
computer even when it is turned off. 

♦Scheduled reports Reports that are produced at regular intervals - daily, 
weekly, etc. 

♦Sequential file A file in which records are stored in alphabetical or 
numerical sequence. 

♦Serial file A file in which records, usually transactions, are stored in 
the order in which they occurred or were recorded. 

Software The symbolic component of a computer system, to include the 
operating system, the data base management system, compilers 
and application programs. 

♦Special reports Reports for which application software does not exist 
when the report is requested and must be specially prepared 
before the report can be produced. 

Success Indicators Need to specify these prior to evaluation of an IS 

♦Supwary reports Reports that use summary measures, such as the mean, the 
range, or the standard deviation, to descrK ^ data in less 
volume than detail reports. 

Syntax error A programming error that violates one or more rule of the 
programming language and may prevent the program from being 
executed. 

System A set of interrelated parts that work together to accomplish 

some goal or objective. 

Systems analysii. The analytical process of determining Information needs 
in an organization and describing an information system to 
satisfy those needs. 

Systems designer A systems analyst who describes, with flowcharts and 

other tehcniques, the programs that will produce information 
needed by managers. 
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Systeits prograiwier One who prepares, frequently in machine lamjuage, 

operating systems, compilers, data base management systems, 
and other systems software. 

Terminal An on-line input/outi3ut device, usually with keyboard entry 
and cathode ray tube (CRT) or typewriter output; a 
flow-charting symbol for the beginning or end of a program 
sequence. 

Timesharing The use of a computer system by several users 
simul taneously . 

Transactional processing The preparation, editing, sorting, classifying, 
sorting, retrieving, and limited calculating of data for 
recordkeeping, report generation, and input to managerial 
activities. 

Trend analysis A statistical prediction technique based on past perfor- 
mance of the predicted variable. 

Top-down An approach to IS design that starts with the objectives 
and/or information needs of either managers or of the 
organization. The focus is on the decisions to be made or 
purposes to be served. It is usually contrasted with a 
"Bottom-up" approach which implies that the system starts 
with existing data or builds upon systems currently in use. 

♦Uncertainty A decision-making enviroment in which the outcomes of future 
events are not known nor can probabilities be assigned to 
outcomes. 

-''Unschedul ed reports Reports that can be produced with existing applica- 
tion programs but are not unless specifically requested. 
Also called demand or on-call reports. 

User-friendly Can refer to either hardware or software designed so a 

person with non-technical background can operate the computer 
system with relative ease. 

UEe**-or1ented Systems which are designed to be responsive to the needs of 
those who use the system. 

Verification The process by which data are checked for accuracy; in card 
input, the keying of data into a verifier for comparison to 
data already p'jnched into the card. 
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SECTION 2 



THE INSTRUCTIONAL INFORMATION SYSTEMS NETWORK 



Progress Repor t 
Attachments 



A, Steering Committee Materials 

B. Initial Mailing to Prospective Members 
^' Newsl etters , Vol.1, Nos. 1 and 2, and 

Directory of IIS Members 
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Progress Report 
November 1984 

The Instructional Information Systems (IIS) Network is a growing group 
of over 100 educators interested in a variety of topics related to instruc- 
tional information systems. The Network includes those involved with test- 
ing, evaluation, curriculum, administration, computers, and drta proces- 
sing. The group has a steering committee, has had a mini-conference, has 
put out two newsletters, and plans exchange site visits. 
Development 

A chronological history of the Network would reveal its antecedents in 
CSE's 1975 nation-wide survey of the work of district research and evalua- 
tion offices. This survey revealed a high level of activity in test admin- 
istration and scoring, and a high level of reporting out to funding and 
regulatory agencies. Concomitantly, there appeared to be low levels of 1) 
existing R&D office activity on test sub-score analysis, or combining of 
such data sets with other existing data sets (such as evaluation or atten- 
dance), 2) an explosion of interest in technology for classroom instruc- 
tion, for administrative services (e.g., payroll, accounting) and 3) a 
growing awareness of the need for data for instructional decision making. 

Following the survey, we completed case studies in districts which 
were, in some way, using test or survey data for internal decision making 
(Bank S Williams, 1980). Further intensive examination of the impact of 
linkage systems in districts (Williams & Bank, [1984a]) led to the 
fonnulatlon of several models of such district systems using management 
information system terminology (Wniiams 4 Bank, [ig84b]). 

() ) 
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As we worked in our "heroic" case study districts, we had become aware 
that district administrators did not conceptualize what they were creating 
as coordinated subsystems. However, when we viewed what they were doing 
from an open-systems perspective, they were indeed creating an information 
subsystem which had identifiable characteristics. 

We organized a UCLA Dean's Forum in the Fall of 1983, to express this 
view, and scheduled two districts' representatives to speak with approxi- 
mately 80 superintendents about their testing-evaluation-instructional 
linkages. At this event, the desire was expressed to create a network of 
people interested in initiating, developing and maintaining instructional 
information systems. 
Activities 

Our next step, tnan, was to create a low profile, non-time-intensive 

« 

format by which people could access one another when and for as long as 
they wished. 

In this particular area of educational change - that is, the creation 
of instructional information systems - practice seems to be leading re- 
search. Change is partly a result of technology push with the increasing 
availability of computer hardwai^e and software. Research is needed to 
document the push, and, perhaps, to guide it. However, what practitioners, 
each of whom sees only a piece of the situation, seem to want at the 
present time is "low energy access to trusted information" (Miles il Lake, 
1975). That is, they want a network which, without taking too much time, 
permits them to use peers as well as academics to answer immediate problems 
of pressing concern and to lay out more long range issues. 

A six-person steering committee met in February 1904 to explore the 
feasibility of forming such a network and to plan an initial set of 

^ DA 
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activities (See Attachment A). They were asked to express their prefer- 
ences for a possible set of Network activities that included: Brokerage/ 
referral; technical assistance/training; information exchange; research- 
development; evaluation and replanning. They were also asked to describe 
what their districts/agencies did. 

From the discussion emerged the idea of a mailing to identify network 
members. This mailing included a Status Survey, an Interest Survey, and a 
note about CSE work (see Attachment B). Fifty-seven percent of the 61 dis- 
tricts responded and indicated high levels of Interest in a newsletter, 
collections of exemplary materials, tips/techniques/tools, working confer- 
ences, exchange visits. Substantively, our respondents were interested in 
matching tests/texts and objectives; with information about hardware/ 
software; with a taxonomy of policy, administrative, and instructional 
questions to ask of the data. 

In June 1984, a mini -conference was held where six presentations were 
made by participants on various topics of Interest to other participants. 
Task groups were formed to address questions such as "What do you do now 
with your existing data base?" "What would you like to know from your data 
base?" "What are some typical questions you are now asked that you cannot 
answer?" 

Two Newsletters have been distributed along with a Directory of IIS 
Networkers on which are more than 100 names (see Attachment C). 

As can be seen from the second Newsletter, site visits are being 
planned for those interested in particular aspects of information systems. 
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Accompl i shments 

Accomplishments clearly depend on the view as seen from where one 
sits. From the perspective of Network members, we have anecdotal evidence 
that the Network has: 

1. introduced to them the idea of Information systems. Information 
systems as a concept is new to most district and school person- 
nel. While schools and district offices collect data, analyze 
data, produce reports, and make decisions, these activities are 
not conceptualized as if they were part of a system. While Indi- 
viduals performing these various tasks may relate to and communi- 
cate with one another during the course of their work, they have 
not, up to now in most districts, thought of themselves as parts 
of an interlocking, coordinated system. 

2. introduced them to one another. Individuals within the Network 
have been in phone contact with one another. In addition, a group 
of district administrators has met to discuss an issue of common 
concern - how to educate the media to a proper and balanced inter- 
pretation of test scores. 

3. introduced them to CSE's work and the work of other labs and 
centers. Although some knew of the existence of UCLA's Graduate 
School of Education, and some may even had had contact with the 
Center, most were not aware of the array of research conducted at 
CSE. Nor did they know of the variety of skills and talents, and 
the technical reports and publications available from CSE. The 
Network, both through the mini -conference and through the 
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Newsletters, Is publicizing the work of colleagues at other labs 
and centers which relates to this area of interest. 
From our own perspective within the Management of Instructional 

Information Systems Project, we have experiential evidence that the Network 

has: 

1. sensitized us to the nature of the work currently being done in 
districts, helped us to convert our ideas into language familiar 

• to those working in the area of testing/evaluation/instruction/ 
data processing, and given us criteria by which we might priori- 
tize our work in accordance with the needs of the field. 

2. provided us with a valued set of colleagues to whom we can go with 
questions/issues/problems/etc. 

3. provided a forum in which colleagues in other CSE projects can 
present their work for discussion and comment. 

Action/Research Issues 

The following is a list of issues and related concerns which have 
surfaced in the mini -conference and have been contributed to by our own and 
other projects' previous and current work in districts. (They are neither 
exhaustive nor yet stated in a form ideal for creating a research agenda. 

The nature of the "information" to be Inputted into an IIS .) 

Comment: The real issue for us Is that we have too much informa- 
tion. How can we digest It to get a better idea of 
where a child is at? 

Comment: We've got a lot of stuff in boxes and file cabinets. 
What, of all of it, is Important to computerize? 
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Cotnment; Could we find out about kids': academic self-concept, 
amount of time they work after school ; what 
extra-cucrlcular activities they engage in; their 
self-report on time spent on coursework; who is at home 
to help with homework; personal problems? 

Comment: We'd like to know how many^f our kids go to college* 
how meny have decided on vocational choices; what 
they're like several years from now. 

Comment: "Who's going to do the work to get what's in our files 
into the computer? We don't have enough cler; time as 
it Is." / 

Thq uses of the "Information" from an IIS . 

Comment: Politics Is the main factor entering Into school board 
decisions. How would an Infonsatlon system help us on, 
for example, school closings? 

Comment: We've been trying to develop a system by which the' 

district can supervise principals so they. In turn, can 
supervise teachers, so that kids' scores go up. How 
could an Information system be useful here? 

Comment: An US could print out reports that could show the 

press, realtors, new parents, what the -school was like 
and what the kids were achieving. 

Conment: Maybe you could use such a system to Identify kids in a 
se#)ndary school who were weak In certain subjects and 
have the objectives of classes be turned into this, Or 
kids who were high achievers. 

Comment: What's happened to privacy? 

Comment: Data can be an embarrassment, maybe even a source for 
law suits. «v *' 

Comment: Is more data an Improvement over informal judgments of 
teachers? Do we know enough, for example, to match 
teacner style to student learninc style? To understand 
the effect of home environment? 

Comment: Can we learn what the causal factors are that account 
* for a kid's growth or behavior? , 
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Attachment A . Steering Committee Materials 
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NETWORKING 

Carol Ttiomas 
Center tor the Stydy of Evaluation 



When people try to solve problems in education, they naturally search 
for other people who are devoted to the same purpose. Networking is a term 
that has been applied to the search to create new channels which link 
peoplfe who can share information in support of school improvement. This 
paper poses a number of questions about networks and provides brief answers 
to them. 

1. What Is a Network? 

There are various kinds of networks, but a key notion is the inter- 
relatedSfess of parts. Miles (1978) defined network in the most abst.-act as 
simply a \et of nodes or points (In social networks, the nodes are persons, 
groups or organizations) connected by lines or links. Combining key Ingre- 
dients of networks he examined, Parker (1979) defined networking as a stra- 
tegy that facilitates the sharing of information and psychological support- 
. among independent Innovators and problem solvers who link together volun- 
tarily as equals seeking assistance not provided by escabl ished^^tems. 
Taking networks for educational Improvement as a whole, their fimbtion is 
to foster the sharing of information and inspiration among Independent 
educators in their local problem-sclving efforts and to assist ia the 
development, adaptation or adoptlor of new programs, products and prac- 
tices. A key factor In networklngyis that Its purpose elicits cornmltment 
and' a sensn of community. Mlles^nd Lake (1975) see the prime strength of 
informal social networks In \\K^T capacity to provide members "low energy 
acc^ss to trusted information." House (1974) asserts that the flow of 
personal communication Is the key to Innovation. Deliberately stimulated 
Interactions among Innovators and problem solvers in some networks remain 
Informal and minimally organized, wMle In some networks the participants 
develop names for the networks and some formal operating p^^edures. 

What A re the Essential Components of a Network? 

Parker (1971^ 1979) suggested that to be successful, interactive 
networks must Include at least these five components: 1) a problem- 
oriented goal with facilitating objectives; 2) voluntary participants 
concerned with the goal; 3) an Information exchange or clearinghouse; 4) a 
facilitating staff; 5) temporary cooperative projects. In addition, Packer 
and other researchers have found the following traits common to networks: 

• a situation of limited resources 

" a sense of being a' alternative to established S|jt«terns 
a feeling of shared purpose »• 

° a mixture of information nharing and psychological support 

■» a person functioning as an effective facilitator 

an emphasis on voluntary participation and equal treatment 

" a sense of community 

" a beqinning with Individual's self-interest 
ERIC '/'^ 
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Potential participants will }}V.eVy be interested not in the p rocess of 
networking, but primarily in the network, as an instrument for achieving 
their own objectives. This implies that the goal of the nertwork must be 
oriented to the felt problems of participants . The goal must be an um- 
brella under which people with overlapping, complementary, and sometimes 
conflicting objectives can gather. ^ 

.3. What is the Role of the Netwo/k Facilitator ? 

All successful networks have. one or more participants who consciously 
facilitate the sharing of information. In many, networks, these coordi- ' 
nators were also the founders of the-^network . A major operational chal- 
lenge facing network facilitators Is the arrangement of opportunities for 
face-to-face Interactions. Since participants in a network have similar 
goals, the open sharing of Information is likely to lead to an awareness of 
'Situations in which teaming could result in benelA* for everyone in- 
volved. The facilitator is responsible 'for tnontforing and attempting to 
maintain the balance bet'-'een give and take among participants, therefore a 
facilitator's sensitfvity tt^ other people is an important characteristic. 
Network facilitators are likely to have disproportionate influence due to 
their knowledge of the network as, a whole and to the special facilitating 
skills thsy develop. If network coordinators begin to use their position ' 
in a manner which threatens the equal treatment of network participants, 
however, the future of the network Is endangered. In summary, an effective 
network coordinator has ^ 

* commitment to the network "s purpose 

** ability to get along well with diverse people 

" a good sense of whom to link With whom 

** a tendency to downplay his/her own expertise 

^ ability to persuade participants to interact 

* a sense of when to be directive and when to be non-directive 

^' What are TypicaV Activities of Networks? 
Brokerage or referral 

Examples: newsletters, handbooks* directories or catalogues, 
rtteterials collections . • , 

* Technical assistance/training 

Examples: process consultation, workshops, seminars 

" Facilitation Of information/insight exchange 

Examples: journals containing articles by members, conferences, 

joint problem-solving sessions, personal /visit exchanges 
' among/membefs 

' Research and development >■ 

Examples: policy analysis, case studicMi of members' work, studies 
on special problems/isf,ues, systonir«tic (levf?lopiiU!nt of 
iPMOvatioris 
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° Political action 

Examples: lobbying, building constituencies 

» 

" Evaluation and re-planning 

Ixamples: surveys for needs/nesources identification, meetings, 
self-study 

« 

/ .5. What are the Factors That May Limit a Network's Effectiveness? 

The following factors have been cited in the literature as barriers to 
I effective networking: 

" failure of the network to meet members' self-interest 

• seriogs ambiguity or differences of opinion regarding definitions, 
intents, division of labor, nature of needs, etc. 

• lacic of follow-through and demonstrated commitment by members 
® problems of turfdom, agency imperial ism 3nd competitiveness 

• tendency to^be overly ambitious or underestimate the time and 
careful planning required 

** Inability or unwillingness of members to participate in decision 
making that affects the network 

• pver-emphasis on money, formality, and visability 
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Agency : 
Position: 



NETWORK ISSUES 

1. Listed below'are some typical network activities. For each activity, ratt; the degree of 
importance for including the activity in this network. Dien -"ndicate the top priority 
activities for your organization. 



Typical Network Activities 



Very Somewhat Not at all 

Important . Important Important 



Top 
Priority 



Brokerage/Referral 

toll free phone line [ ]• [ ] [ ] . 



newsletters [ ] [ ] 

directories [ ] ^ ..... [ ] 

catalogues . . .\ [ ].*....[ ] 

referral by mail [ ] [ ] 



[ ] . 

C 3 . 

[ ] . 

[ ] . 



materials collections [ ].'....[ 1 [ 1 ^ 



. C ] 

. r. ] 

. r ] 

. C 3 

. [ 3 

. C 3 



Technical /Assistance Training 

long-term consultation , . 

workshops . 

seminars/study groups . . 



[ ] 
[ 3 
[ 3 



[ 3 
[ 3 
C 3 



[ 3 
[ 3 
C 3 



Informajtlon Exchange 

co-author journal articles 

conferences 

joint problem-solving 

sessions/task forces . 
personnel exchange .... 
exchange of visits among 

members 

trouble shooting . . . . 



V 3 
C 3 

[ 3 
C 3 

r 3 

[ 3 



[ 3 . . 

[ 3 . . 



. . [ 3 
. . [ 3 



[ 3 
[ 3 



[ 3 
[ 3 



C 3 . . . 
13... 



. . [ 3 , 
. . [ 3 . 



Research and Development 

case studies of members' work. 

studies of special issues, 
e.g., policy /management/ 
procedures ........ 

development of products and 
practices 



i; 3 
r ] 

[ 3 



L 3 



• • • L J • 
. . . [, 3 . 



[ 3. 

[ 1 . 
[ ] . 



Evaluation and Re-plannint^ 

survey's for mH^ds/ 

resourf^es identif 1 cation » . 



. C 3 
. C 3 
. C 3 



C ] 
C 3 

[ 3 
C 3 

C 3 
[ ] 



. . C 3 

. . [ 3 
..13 
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What would you/your agoncy like from the network? 



4 



l^at could you/your agency contribute tO'thb network? 



Name: 
Agency : 



Posi tlon: 



INSTRUCTIONAL INFORMATION SYSTEMS ISSUES 



Instructional Invormation 
System Components 



Users {e.g., assessment of 
user ne^ds; specified 
decision focus; delivery 
processes/reporting of 
data to users) 



Your Organization's 
Current Situatipn 




Important Issues for 
Your Organization 



TO 
cn 



Inputs (e.g., What data 
Is in the system; 
i nstrumentation; 
what analysis is done?) 



Outputs (e.g. , how, 
when and by whom is 
the system accessed?) 



S uj[)po r t s (e.g., supports 
for Information-based 
"decisions; coordination; 
role of evaluation office; 
role of staff development) 
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l (K;il1tif^s hardv/aro; 
software) 
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Attachment B. Initial Mailing to Prospective Members 
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INSTRUCTIONAL INFORMATION SYSTEM NtTWORK 
INTEREST SURVEY 



Are you interested in being involved in an Instructfonal Information System, 
Network? 

YES ■ ' ■ ' ' ■ 



MAYB.E, depending on 



What would you like to see an Instructional information System Network do? ' 

(Please check all items that interest youj dbuble-check the three most 
iuiportant. ««l^d others.) y * . 

Information Exchange 

toll-free phone line % 

newsletter , 

" special Interest telephone directories 

resource catalogues 

collections of exemplary materials 

^ tips/techniques/tools 
. seminars 

working conferences * ^ . 



Technical Assistance/Training 

workshops 

personnel exchange 

hot-line trouble shoot-lng 

long-term consiH tation 

excharjge of visits among members 



Research and Development 

case studies of members' work 

issues papers 

development of guides or manuals ^ 

co-authored journal articles 

' co-authored professional association articles 



Other (list) 



» 



Name , School /Hi 5; trie t/Agency Your Position 

Address • , , , Phoi.^^ 
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INSTRUCTIONAL INFORMATION SYSTtM NtTWtJRK 
STATUS SURVEY 



(,s»,/ik;i.a 

3/84 



From tiis brief survey, we anticipate developing a listing of what interested individuals 
arid their organizations (that is, schools, districts, intermediate agencies, etc,) do in 
tlie area of instructional information systems. . ■ 

Would you be willing to have your name included on such c ^ . A along with a brief 
description of your organization activities? YES _^ NO 

Would you like to r*eceive from us a directory of people interested in a network with 
a brief description of their activities? YES 



In our organization, we now 



(c 



ec 



all that apply) 



We'd like to get .iwre information about 
how to . . . (check all that apply; 
double-check the three most importa-it) 



1. adm1n4stpr nonn-referenced tests . . . . 

2. administer criterion-referenced tests 

3. collect non-achievement data (surveys of attitudes, 

. . . . school climat^ etc.) ^ 

4. have a data base that includes student ID, test information, 
.... and non-achievement data 

5. know how our tests match our curriculum and our text books . 

6. have a planning cycle that feeds data into instructional 
.... decision making • • • • 



7. have a way to use data for policy purposes ... 

8. have a taxonorry of questions to ask of the data > 

9. provide different information to different users 

.... (teachers, principals, parents, board) , 

10. have ardsl^very system for reporting data to different 

. . . groups of users 

11. provide staff and board orientation to data interpretation 

12. coordinate theVai^'gement of evaluation, staff development. 



and instruction 

13. computer analyze our data 

14. use customized software for data ana^sis 

15. use commercial software for data anilysis 

16. have administrator-used hardware/software at school sites » . . 

17. have district computer facilities with terminals at local sites 

18. have district-school computer facility ........ 



19. provide tnnlning in administrative cotnputer use for 
. . . . . teachers and edjninistrators 



20. have policies regarciiin arc and confidentiality of data 



. [ 1 
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Name 



Ik hCiOl/Di strTc t/Agericy 



TOfi 



Your Pof.itiun 



FAR 



■»*r* ■> » - 4.-. 



An Evaiuatinn of 
Montgomery Courtly 



■ tM.-.w '-^nf ffv»-y *rTif% 4!'. JiX^n;j 



•*h?t u>cd pri>|;*ni k^\h:<5 ^v! ^tlic i^-fr<: 
P'*:*iU*m,. fuiwr^i*?, ^ur<fv^vK\ ^^M^m^irti 

/v>>j(^om ■ i/t ^^^^ Af<?nf|^;imjr>y i oufiti 
^^ffi,; ^^iNh:' "^^tW Sifr^tfm, ^^^mtp 

T»>f. f'^r'pu'ryrf ■\\Vv^^J^ruy fV^>iir\;tv. 

hat t loK Testhicj 



. '">>;■>' ,r^-!;"^ 



/.^mifMhf «f\>'*»f'*it (iJiKliJiv 

MWi^Hit pwfKlt^y Ir^U Wiih 

mir.j%irtKirH, (r«cKrr> puy^nU: vru^ienH 

f ! >iK /{tfrVfi5>f^^*' imuuv JiH)ri4i moilf^ 
h uipp-ci'nrni j wopr and \f 

> n^:Ar^;»U *^;jf ip^r-f^ t^^^^ t'^'. 

* . • t >«| I U ^ii^ 1 . J ■ • *V ' , ' r'^ ^ : ■ ^ . ■ ■ ^ 
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V<V!.l, no A 1"'^ May, 1984 

A WE VSIETTI^R AB OUT IHSTR UCTI ONAL IKFP RHATIQN SYSTEMS* 

NETWORK NEWS 



We lcome t o 
education 



system (IIS) 
iclosed with 



the new IIS Ne t work , Many of you from districts and other 
agencies responde<l positively to our CSt: survey and indicated 
that you were interested in being involved in an instructional information 
network. A listing of those individuals and agencies 1:. 
J brief overview of the survey results. 



Network Newsletter - Most respondents were interested in having a rtew? 
letTOr to shar<» information about their activities. With your help, CSE 
staff will take the responsibility to produce and mail a quarterly news- 
letter (nothing fancy!) during 1984-85. In order for the newsletter to 
reflect networkers' interests and activities, we need you to submit items 
in categories such as NEWS, GOOD IDEAS (programs or projects that your 
agency Is proud of), HELP WANTED (issues or questions you would like to 
discuss with others), RESOURCES (items about papers, materials, guides), 
and NDTEVORTHIES (personnel news, conferences scheduled, expertise to 
share). So when you have Information, materials, ideas, projects, etc.. 
to publicize, please call Adrlanne Bank. CSE.||213) 206-1526. 
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Oune Mi r^i -Conference - To kick off the network, CSE is planning a one- 
day conference for June 20. ^984, at UCLA from 9:00 a.m. - 3:00 p.m. 
At this conference there will be: 



Raggedy Beginnings 



opportunities for networkers to network about 
interests/activities 



An Hour of Overload ~ six 10-minute high-interest presentations 

packed into an hour (tin-^ afterwards for 
discussion) 



Ir !v1d1 and Not So 
Trividi Pursui s 



Arrangenient*'. dnd 
Next Stop 5 



sniall group sessions: raising questions for 
and about question-driven instructional 
information systems 

expectations/ joint projects and 
papers/newsletter, etc. 



?■ ! thfT brinq your Umch or you cvin ordfr froin the sandwich h-!r» We' ll 
I i»rovid»? drinkv. and nibble;,* Free parking will he iirrancjed, 

I > Of iMrthor dctdli!, dv>6 direction*; rfMd p- <]('. PH'd!;<' call m; or 

j 'Urtil hmM the RSVP fornt by ,,lunp \?U\ if yov m-<.' coininij. 

"ProflM'f fl hy Mi»naqement. of Instruc tloo.i! Ififonndtiinn '>y'>tems. Projt'tt , 
(;St/UClA.. " n I 
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GOOD IDEAS 



Academic Achievement Awards - The OXNARD Elementary School District has 
developed an effective academic awards program to provide recognition to 
achievinq students. For information about this program, please contact 
Dr. Fer.»dndo Elizondo, Oxnard Elementary Schoo^ District, 831 South B 
Street, Oxnard, CA 93030, (805) 487-3918. 



Hi gh School Entrance Exam - NEWPORT-MESA has Identified several profi- 
ciencles that all eighth grade students must master before entering high 
school. Those who have not passed an "entrance "ex«mi nation" on 1:hese 
proficiencies can attend a special summer school program. This plan has 
received wide-spread Interest and support throughout Orange County. For 
more information contact Dr. Dale Wooley, Director of Research and 
Student Services, Newport-Mesa USD (714) 760-3295. 



Social Studies CRT's - NEWPORT-MESA has for many years been a leader in 
developing a system-wide competency-based testing program. They 
recently completed a competency test for social studies which is being 
used for high school entrance and graduation purposes. For more 
Information contact Dr. Dale Wooley, Director of Research and Student 
Services, Newport-Mesa USD (711) 760-3295. 



Curriculum Cluster Analysis' - SAN DIEGO USD has developed a system for 
clustering MKT results around curricular problems. For^ example, lan- 
guage arts Items having to do specifically with capitalization or punc- 
tuaticn are clustered for each school site. This allows each school to 
target Instructional activities towards improving specific, identified 
problem areas. The principals and teachers have responded very favor- 
ably toward this practice. For more nformation contact D^. Grant 
Behnke. ^ssoc1ave Director, Evaluation Dept., San Diego USD (619) 
293-8509. 



School -Site Data Histories - SAN DIEGO USD's Evaluation Department ^fls 
developed a comprehensive test reporting system that displays test data 
and other data for each school using histograms and o|her visual tech- 
niques. These data are indexed and Cross- referenced in notebooks that 
are prepared for each school site. Data can be accumulated from year to 
ye^»r. These notebooks then contain a "data history" for each school 
site. For more information contact Dr. Grant Behnke, Associate 
Director. Evaluation Dept-, San Uiego USD (619) Z93-ab09. 



1 12 
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Test/Teaching Alignment - NORWALK-LAMIRADA USu has a matrix showing the 
relationships among the CAP, CAT', district iroficiency tests and dis- 
trict skills continua. Teachers use this to identify those skills which 
are tested liut not taught. For more information, contact Dr. Betty 
Coogan, Assistant Superintendent, Norwalk-lal^irada USD (213) 868-0«l. 

I 

Improving SAT Scores - ANAHEIM UNION High School District is developing 
teacher-designed SAT test-taking materials for bpth students and 
teachers to be used in \]^g)r\ school English classes. SAT scores have 
improved dramatically. For more information, contact Dr. James Cox, 
Director, Research & Evaluation, Anaheim Union HSD (714) 999-3558. 



★ ★ * * 



HELP WANTED 



Anyone have good ddta display techniques ? 

POMONA Unified School Distric t is Interested In how other districts are 
providing/displaying data Iparticularly longitudinal data) to various 
groups. They would like to see examples of charts, graphs, or Just 
effective, clear writing, and talk to others Interested in this prob- 
lem. Please contact Mike HartJnan, Director of Program Assessment, 
Pomona Unified School District. 809 South Garey Avenue, Pomona, OA 
91/66. (714) 623-5251, ext. 404. 



What are you doing about effective schools ? 

OXNARD Elementary School pi strict Is focusing pn variables that the 
literature says increase school effectiveness, particularly school cli- 
mate, high expectations, and frequent monitoring of student achieve- 
ment. They are developing training for administrators. They are In- 
terested In Identifying w^ys to measure the effectiveness of fre<)uent 
Monitoring of student achleveMent, and In how one can translate "high 
expectations" Into an Inservlce prograw, e.g., what are the skills and 
competencies one needs in order to "irlng about high expectations. 
Please contact John Marshall, Assistant Superintendent, Educational 
Services, Oxnard Elementary School District. 831 South B Street, Oxnard, 
OA 93030, (805) 487-3918. 



Software for teachers and principals? 

NEWPORT-MESA USD has received a grant to develop software for training 
'teachers To use microproce' sors. They are Interested in current 
practices regarding how principals use microprocessors for school-based 
instructional aanagewent. Anyone with infonnafclon about this should 
contact Dile Wooley, Director of Research and Student S«rirvices, 
Newj)ort-Mesd USD (714) 760- 3295. 

I \ :i 
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RESOURCES 



District Policy on Microcomputers - Schols are acquiring microcomputers - 
at a phenomenal rate. There is great variat'ion, however, in the ways 
these computers are used and the district office's role in introducing 
computers into the schooling system. Adrianne Bank, Richard C. wmiams 
and Carol Tho>as of CSE have written a paper that highlights some topics 
with which all districts must eventually cope: hardware and software 
acquisition.lmanagement systems, curricular and staff development pro- 
grams and ir<kfuctional methods. The authors propose a contingency 
approach that suggests that districts' planning be ongoing, incremental, 
adaptive, and self -correcting. Four components of a contingency 
approach are discussed: 1) conducting a situation audit of external and 
Internal environments; 2) generating support; 3) formulating district- 
wide policy; and 4) developing an ongoing operational plan. Anyone 
interested in a copy of this paper, please contact Katharine Fry - CSE, 
145 Moore Hall. UCLA. Los Angeles, 90024, (213) 206-1536. 



Var ious Perspectives on Information Systems - A number of CSE papers^ 
were presented at the 1984 meeting of AERA which approached the topic. of 
Information systems from various perspectives. Among them: 

Bank, A. What's In A Name? Defining District Information 

Systems 

Burstein, L» Information Use in Local School Improvement: 

A Multi -level Perspective 
Sirotnilt. K. Using Versus Being Used by School Information Systems 

Williams, R. Relating Instructional Irtformation Systems to School 

District Decision-Making Structures 

Anyone Interested In obtaining copies should contact Katharine Fry, CSE, 
145 Moore riall. UCLA, Los Angeles, 90024, (213) 206-1536. 



* ★ * * 



fiOTEMORTHIES 



IDFA Fellow P rogr<^ m - John Marshall. OXNARD School District, has been 
erte?*lo"AHencf'"the IDEA Fellow Program, sponsored by the Kettering 
Foi!ridation» o be held in Denver, July 7-13. 1984. This prograin bring 
selected admif istrators together to i>hdre cammon projects and problems. 
;..ohn ii interested in bringing instructional Information systems issue*, 
to the' meeting. If anyone has sped tic concerns they want to di!,cus»v 
with John prior to the meeting, call him at (605) 4B?'3*)Ui. 
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Ripple effects - As a result of CSE's October 1983 Dean's Forum on 
linkin"g Testing, Evaluation and Instruction, Jim Cox (ANAHEIM UNION HI ah 
School District). Lois Bl^ackmore (GARDEN GROVE), Bob flyan (SANTA ANA). 
Joellen Crawford (PLACENTIA). Don Hays (FULLERTON) and Totn Martin (ABC) 
have been having agenda-less meetings every six weeks or so. Last time 
th6y met with education writers of local papers to discuss reporting of 
CAP scores. "We didn't try to resolve anything - ;just make contact." 
Jim said it paid off later when a reporter called to ask for his 
opinion. 



Management of Instructiona'! Infonnation Systems project at CSE, in addl- 
tion to facilitating this IIS Network will be developing a paper on 
lessons for educators from the Management Information Systems field, 
will be developing a taxononv of questions which policy-making 
administrators and teachers can use to address data in IISs, and will be 
making presentations around the state. CSE also has a number of other 
projects touching on infonnation systems, particularly the Systemic 
Evaluation project directed by Leigh Burstein anc Ken Sirotnik. tor 
further Infonnation about all this, call Adrianne Bi.nk (213) .;206-152!^ or 
Dick Williams (213) 206-6639. 



* * ★ * 



ease — - mail back, your comments and suggg s tionSt even if you can't come 
to the conference (see other side for iiddress). 
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RSVP Form 
June hfini -Conference 



Please call us {213)f206-1535 or return this form by June 12 to Katharine 
Fry, Center for .the Study of Evaluation, Room 145 Koore Hall, UCLA, 405 
Hilgard Avenue, Us Angeles, CA 90024. 



For parking, enter campus on Westwood Borleva'^d. Stqm, at the Kiosk and 
ask for. directions to Lot 6. "t^^ 



For the meeting, go to Ackerman Hall 'the 1^ ail ding where the book store 
is), take the elevator to the 2nd floor, go to Room 2408. 



t If I 



.....J L, 




c 

> 




7 r~-J^.:,. -Kljf'y 



rx iz X -St n n 



(tear off, mall back) 

The following personCs) from our agency will attend the mini -conference: 



Name 



TTtX 



TTW 



r 



mi? 



AfjVfhcy '~W\ sirTcl 
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parking space',. | , . , }3I: h'"''^ \ 
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OVERVIEW OF NETORK SURVEY RESULTS 

We received a total of 37 Indivi'dual responses tg the survey (a 57% 
response rate) from 33 different agencies. 

Preferences for Network Activities 

Network activities that most respondents expressed Interest in in- 
cludecf: a newsletter, collections of exenij3lary materials, tips/techniques/ 
tools, seminars, working conferences, workshops, exchange of visits among 
members, and the development of guides or manuals. Most respondents we**e 
not Interested in a toll-free phon^ line, personnel exchange, or long-term 
consultation. 

Scventy-fiv^e percent (N = 27) of respondents indicated that they would 
• like to have their name and a brief description of their agency activities 
included in an IIS directory. A total of 86% of the respondents (N 31) 
indicated that they would like to receive such a directory. 

Current District IIS Activities 

- Over 70% of institutions " responding administer norm- referenced 
tests, computer analyze data, and provide staff and board orientation to 
data Interpretation. - Over 60X collect some type of non-achievement 
data, know how their tests match their curriculum and textbooks, provide 
\ different Information to different users, and have established a delivery 
system for reporting data to different groups of useres. - Fewer than 
42X have a way to use data for policy purposes, use commercial software for 
data analysis, or have district computer facilities with terminal at local 
sites. - Fewer than 20X of respondents have a taxonoiny of questions to 
ask of the data. 

Agency IIS Interests * 

Our sample of school districts and other education agencies indicated 
that they would like to get more Information about: 

• how our tests mat i our curriculum and our textbooks; 

** a way to use data for policy purposes; 

a taxonomy of questions to ask of the data; 

** delivery systems for reporting data to different groups jf users; 

" coordinating the management of evaluation, staff development, and 
instruction. 

Ttie following Table suminarlzes the information v/u rfMirivn' from c>«jf3n(;if»s 
about their current activltes arKi interests. 
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collect m>o- •chleveaettt data (lurvi^s of 
•tiTltidM^-icl^l clliiite» etc.) 
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10. 
11. 
12. 

13. 
14, 

16. 
\h 
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users (teachers, prInclpaUp parents. bo4rd) 
have a delivery systea for reporting data \q 
to different groups of users 
provide staff and board orientation to 
data Interpretation 
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staff develofnient, and Instruction 



cuN^uter analyse our data 

use customUed software for data 
analysis 

use coMRprclal software for data 
analysis 

have a<lalnlstratn!r usrd hardware/software 

at school sites 

have dhtrlrt romjter faculties wit i 
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nntrles In this table Indicate. \n general ♦ district activities and needs for lnfor««tl<m. 
Drfflnltlons of a^tlvUifS uf course, vary by district and Individual Interpretallrjn. The^e 
mtrl#»s shcxjld not be reg^ dri as rtif^th data or cUed In any way. 
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A MEVSUTTSU ABOUT InsmUCTIONAL IHFORMATIOH SISTBNS 

Vol.1, tio.2 Oatobev, 1984 



t^R^Txrs w^T\Aio(x tmimu LEWS TO ?Lftn« Ft>R. f miter 



Mini'^Conf erenow at UCLA 

The flr'^t ever Mini -Conference on 
Instructional Information Systeins 
was held at UCLA on June 20 with 30 
district and university people in- 
terested in policy, administrative 
and classroom uses for IIS: 

The morning sessions featured "An 
Hour of Overload* — six 10-minute 
presentations on current develop- 
ments related to districts' use of 
In format Itni systems. The presenta* 
tions included a description of 
software for administrators, expla- 
nations of processes of developing 
computer based information systems, 
a procedure for matching tests to 
curricula, a way of training raters 
to grade essays, an* the dev«*Jopment 
of a student information survey. 
[See pages 4-6 for brief summaries 
of these presentations.] 



!'> your district doing something to 
'^ink Information (tent scares, 
.ntf^ndance patterns , sotlal data) to 
instruction? Pleaiie )et us hear 
from you so we can help other in- 
tpr<»stert districts connect wUh 
vou. CaV^ Katharine Pry it 



Future Activitieo 

Participants at the June IIS Net^rk 
Mini -Conference said they wanted to: 

- Continue to meet; 

- Visit a district where **someth1ng 
good* 'Is going an; 

- Have CSE play a brokering role In 
bringing people together; 

- Distribute a directory of dis- 
tricts and di strict personnel 
involved with IIS. 

- See liemonstratlons of software 
actually in use for IIS; 

- Learn how to selKit and Interpret 
dewografhic trend Indicators for 
strateijtc planning; 

- See displays of simulated data; 

- Describe evaluati m-dri ven school 
Improvement systims for princi- 
pals; 

- Demonstrate practi al uses for 
data; 

- Discuss how to oiivelop and use 
data bases; 

- Define problems ijis* icts had in 
implementing d-n nases wUh the 
^>olution strategl s they uiied. 



As a st(^'= V'r^^r-x connect! nf| IIS 
people nith one *noThe^, w»> hjve 
'^ent long with thu ^ssue our 
Mews^f*tt<*r mailing Hst. P*ea'.e 
tell us whi] *^1*>e nhuuHt>^^ 
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Have you heard about Microcomputers 
Applications In Education, the Spe- 
cial Interest Group (SIG) of the 
American Educational Research Asso- 
ciation (AERA)? . 

Their infonnatlve newsletter says 
that the purpose of this SIG is to 
promote and facilitate the sharing 
of information among people inter- 
ested in microcomputer use and re- 
search. Major areas of interest are 
(1) teaching about microcomputers 
and their applications; (2) teaching 
with microcomputers; and (3J conduc- 
ting research, evaluation, assess- 
ment, and management/administration 
tasks with microcomputers. 



The group's members include college 
and university faculty, local and 
state level professionals, and re- 
searchers. Membership in the SIG 
gets you a newsletter that features, 
news of the group's activities, 
reprints of articles ancLa "bulletin 
board" for exchanging information 
about Events. Products, Software, 
Research, Positions and Personals, 

Yearly membership fees are $5 for 
AERA regular members und $8 for 
non-members. For a membership 
application call Dennis 0.eck. NWREL, 
300 S.W. Sixth Avenue, Portiand, OR 
97204- (503-248-6800). 



DO '^mi sire vt^rti T/jmssr v«u ? 
voice tv^ii^.te 



CSE win serve as /broker" fn setting up^slte visits so Network tnembers c^jn see 
f1rst-l«nd what other districts are doing. If any of these site visits Interests 
you, can Katharine Fry at CSE (213-206-1536). 



Q >'orwa1k-LaM1rada USD win demon- 
strate how they use an evaluation 
matrix showing the relationships 
among the CAP. CAT district profi- 
ciency tests and th.^ district skills 
continua. Tlie rxperlerrces that 
teachers, principals ''and central 
office staff have had with the sys- 
tem win be presented. 



U flewpnrt-«esa USD has an enrol 1 - 
ment prediction »ystem on a ma^n 
frame computer. The district has 10 
years ejcpeHence wi tH this system 
f^h^cH Is espec^aMy useful \t^ prf»^ 
d^c:t1nq decUnlnq enrollment. Thf* 
«iynt«*m can also be used to simulate 
ft'^e clos^^a of nchool stter the 



0 Santa ^tonlca Sctioo^ District is 
trying out a school -based instruc- 
tional infonnatlon ^ystew with Apple 
lit- Data about student achieve- 
ment, attendance, language skills 
and home background will be avail- 
able in tne principals' office on 
computer . Pr 1 nc1 pal s are work \ ng 
with CSE's HIIS project to develop 
,4 list of high Interest questions 
about the school, diss functioning^ 
or iqdl vidua! ^itudenf; to *?nsurp 
;:hat the system w^n be uspfuT . 
Three schools have terminals. Com* 
visit with pr'jgram devnlopers, pr^n- 

c lerks 



ERLC 



114 



/ 



At a CSE sponsored symposium held at the Evaluation Research Society/Evaluation 
Network (ERS/ENet) meeting in San Francisco^ Walt Hathaway (Research, Planning A 
Evaluation, Portland, Oregon Public Schools) and Peto Idstein (Gypervlsor of In- 
struction, Christina School District, Newark:, Delaware) both made impressive presen- 
tations. Hathaway described the hopes school system administrators have had for 
con^rehensive information systems, the obstacles to achieving tnese hopes, and new 
conceptualizations and technologies to overcome them. 



ERiC 



As Hathaway sees it, the hope is 
that "by collecting, analyzing, 
reporting and using accurate and 
appropriate information about stu- 
dents, and about classroom, school ' 
and district support systems, we 
could help create self-renewing and 
ever better learning communities." 

The obstacles in the way of achiev- 
ing this hope include: 

- The dominance by the 'business* 
functions of the district over 
the resources committed to data- 
based instructional decision 
making* 

- The shortcomings of extant hard- 
ware and software systems. 

- The lack of theoretical and 
causal models of lear^nlng. 

- Andf perhaps most Important of 
all, the difficulty of enlisting 
the sustained coiwr(t»ent of over- 
worked school teaching and admin- 
istrative staff to the difficult 
task of gathering, maintaining 
and reporting ^'upward'* suffi- 
ciently accurate data to support 
deci s i on^mak i ng needs at 1 evel s 
further un the educational 
hierarchy. 

Hathaway *s optimism about comprehen- 
sive information systems stems from 
the n^fw concept (5T distributive pro- 
cessing and the new technology for 
school and classroom-based networks 
0^ micro and minlprocessors linked 
to fhe district's central proces- 
sor. This decentralized system 
a Hows **the data upor wh^ch higher 
^eve^ decisions are biised . . . itol 
createliby the school u«^<?r prima- 
rMy to wet his or her owm <leci- 
^lon-rnaking needs > . 

Hathaway %uqq*'^t% sevB^^? n^p^ ^^tr 

nip<»ds, 'ihfHtji's, rp?«rntness ind 
^»«rnern - 



- Grouping and [ilacing students so 
that their learning needs may be 
met most effectively and 
efficiently- 

- Better and easier classroom, 
building and district management 
resulting In better decisions at 
1 ower cost and more time and 
other resources spent on instruc- 
tion and direct learning support 
and leadership, 

- Prompt, accurate and thorough 
evaluations pf program cost 
effectiveness. 

- Timely and thorough data to e 
support , policy fomulation and 
monitoring. 

For a copy of the paper presented at 
the meeting contact Walter Hathaway, 
Portland Public Schools, 50i Mo. 
Dixon Street, Portland, OR 97227 
(503-249-2000). 

Pete* 5 presentation focussed on the 
pi tfall s and errors that occurred 
when a school district with 15,000 
students and 25 school buildings 
infil*iiienttd in electronic infonaa- 
tlon management systeni* He discus- 
sed nine areas where problems cart 
{and did) occur. They were: 

equipment (e.g., operating systems, 
boards, terminals, printers an* card 
readers) , appl ications software, 
data base management* protocol con- 
version, communications , handshak 
^nq, handhoVding, redundant tnureau- 
':r«c^**s» and thi» ' ?ary pf *-cept v?*'^ 
of the world by end )%ksr%. 

By the way , Pete has formed ^ 
subgroup of the Microcomputers *n 
Eflucatian 'ipec U5 Interest 'jrnup 
(sr-e article, p. ?! 1ev<^jned ^or 
school peop^f?. He ^opes that mefl*- 
f.wr% f^iM sharp ^nti^rp^rs ytt^ 
^f^ii *ge^ toqethi:»r sf}. our vu^9r^^u^) 

f'*r«,f tr) Jfh^r ;^.nOpU* ^hn. 

itiyrk ftvf 3 'H /tnt;.' 'ie^d your ^*ird'^ 
'snd ^ i> t t!f r ^ * u*^ \ -r, i^' " . 

'iChoo? 3Ht.r^, Sf*w^r^,. ' 

t22 



MINI 'Immt^iCc Ktf^H ■ I4«r5 



v^{f'*<v<« tfsf tSf pro^et't *r:tc,ft\h# aod K^n s^ro^^ntit direct 



toy*' Hi^h School in S<mj 



t*»fl'S'T(# on- tine «iifoinMtlon 4y»t«« 
,s-,^t.r Tt»if 1% «nd SIP support that 
'/•c-^aiKj itu<flleflt 5»cic<>rouftd anfl r*- 
*<»rr«! »ftfoni»tlon, gr«iSe recor<ii. 
^,*uUtw« l«n KOre, «tttn<t»ncif 
toww «icn«dol«», four-yeftr 
coufit {)'«n<, and care«r«lo-$ltn ? 
3113 ski* H'tQ-cour^ti "iipi iflttrHjetJ 
'or *«t part of th« K^OOt 1, 

.a»-«*f' •M(|tifi ?cnoo' Program. Thin 

'If-'fl At thr »rt<Jiv*<Jui1 , 

::•*?» fjrt !>n s iay f^i *..at* Hay 1^4, 



r> r 



:f««p'*t*d tf?*" 



survey *tt»t less tnao 2*^ Hatred data 
for «»»t questions- A snorter and 
5,t^gntly '»:)difled version sdain- 
?5te»*ed to mcQWtng lOth grade stu- 
lisntj during a pre-sc^ool orienta- 
tion •tttlng with counseiors A 
f^vUed surv#y H trlanni^ for Spring 
to aonUor trends m student 
attUudfiv 

TTte Survey resuJts nave already been 
jsed <)y ichool staff in their re- 
quest for S88U funding and tn docu- 
•entlng a recowaendatlon for speci- 
fic staffing decHfons. The survey 
data are atio a centraJ part of tnc 
fjevetop«ent of f.tudent- 1 e veK cUss- 
^e¥«?l. and school *?eve> reports in- 
tended t« routinely Inform decision- 
making By teachers and adulnlstra- 
xjQr\ \n their school smprowewent 
efforts. 

<»ef w\ wisning to rece*v«? copies of 
est reports on the %>yal-CSE 
CO VI aborati on or copies of the 
ju^ty Jnstrtawts and reporting 
'oriMts developed as part of the 
project should contact leigh 
Huriit«<n fn3-8?5-i88«?> ar Ken 



■ "I 



S \t tt r « *• 



1 (• ••: ? 



i 



• iMt*-^,:* <flH^twi*f^Ud d 'i»igthy ^>rof t?«im:y tetvt ^nd netdi^il a system ti:^ 
■li»ta^#f^l<»vii) 5x^tw 50on «5 {>oi;$1b1i». Th«»ir ^hort^-terw solution wikt; 



30 w^^Ti ^ il» C^O<t>!^ 



Z^rni ^'homas of SWRL dfscr^bed the Dt^^trict Executive ftrow^ilnq System (DKBSK a 
nf^tware pac«^dge designed to give an educatio' executive ready access to school 
r jnwdtion cp^P^^^Bble to the t?usiness information systems avail ab^^ to private 
^iector e)tecut1ves. 



DEBS U deslgfied to provide "inform- 
tun to answer dUtrict school ! rtg 
finest 1 0f c5 In thre# areas: 

The Instructional Acco«p1 <55h«ients 
Inforwitlon Hie penults the user 
to look at the fnstrucilonai ac- 
compi tshmnts of students by sub- 
Ject» grade » skin area* and/or 
school . 

- TVte Oeiaographic Information File 
contains infanwtlon relevant to 



the characteristics of 4 particu- 
lar school (for exdmpie, ethni- 
city , language, fluency, and 
gender) . 

The Instructional Pr .<i,t1ces 
Information File contains Infor- 
mation relevant to an Individual 
d1$trict*s program, sucn as text- 
bodk series, aaiount of of f -grade- 
Ifvel Instruction, proportion of 
textbook coinpleted, absentee 
rates. 



IXS Opi^'ratxfig ProblgmB Idiintifi^d 

wmie *^^^rray, CS£ Visiting Scholar, described a district which Improved its Infor- 
mation quantity and flow but unwittingly created ^aajor data entry and retrieval 
problems. Murray speculated that these problem were caused by lack of communlca- 
t on with uters and designers, a<(l»ln1 strati ve boundaries, and lack of long range 
planning. She suggested that creation of ^cng-term evaluation of system processes 
and products along with close monlturl^^ system utilization wpuld help avoid 
these oroblems. 



JD^n Herman of C5E described a dis- 
trict who thougnt their CAP scores 
14^^ too 'ow« est "Sooked at the 
match between CAP and district 

objectives and founts that over 50 
i>f^rrent of the CAP terns ^rv. not 

^ener^i conc'iu:Slon^ f^om this 

:. Tp-sts a^'^e vKCur^te ^nd appropr ^ 
^rc ^ntiicator^i of ^tij6pnt progn»s*^ 
ifvd school f>::t1 veness cnVy under 
'! *;p^r^al circiiwstance: »i^hen i:hf^ 
fr< T. r^f^PcV the ^as^r <|OArH an<t 
t;"-. w^^cn tHp 'ichorj^ ( *"» try-' 



ula: Do Thi^if Match: 

I. Students do not do well on con- 
tent to whffch they have not been 
exposed, and districts likewise do 
not look good on tests covering 
content which Is not p<^r r of their 
currlCHlttfn 

Hem^an suggested thc^t once intnct 
goals and objectives are defined, 
there are two basic strategies to 
finding an appropriate test with the 
rfeslrabiUty ^nd resou ces requlre^l 
for r^ach strategy Inversely related^ 

t> Devela^ te^sts to matcJn your dts- 

ir^Kt curriculum; i;*r, 

J ) - a f y z tv/ a 1 ^ ? f» t e *i t .1 ^ f i 



Analytia 4»o*a»m«nt of Studenta' Expoaitovu and dafrative Writing 

Jaiiws Burry. CSE, described the development of analytic rating scales. Burry ex- 
plained that the CSE project developed analytl- scales to assess students' writing 
because they have greater Instruction?^ payoff than holistic Judyn»ents. There are 
separate expository and narrative scales because each mode requires its own writing 
skins. 



The scales can provide teachers with 
a profile of students' specific 
skills in: paragraph organization, 
support, essay coherence, grammar 
and (nechanics. Tliey also consider 
general Impression inhlch considers 
originality and tone In addition to 
the other writing elements. 

Each element described above re- 
ceives a score indicating mastery 
(4-6) or non-««stery (1-3). Each 
essay atso gets a total /Averaged 
score. 

Analytic rating takes two to four 
minutes per essay depending on 
sample length, compared to a minute 
or so for holistic judging. Tlie 
additional analytic time gives 



teachers* useful information with 
which to plan instruction around 
diagnosed strengths and weaknesses. 

Training of teacher raters take" 
several days depending on: number ot 
trainees; how quickly they accept 
and consistently apply scale 
criteria; whether training Includes 
ratings of actual student work. 

CSE Resource Paper No. 5 describes 
the scales (which are CAP'S 8th 
grade writing assessment measure), 
the underlying research, and the 
training (Quell malz, E., S Burry, 
J.v Analytic Scales for Assessing 
St udentT" Expository and Narrative 
Writing SHMs . ITO. 



vjrve HEflo ft?6i^r # * 



A It^w Paokage for ApplM IIS 



A Go 



We've heard good thit }% about 
Applcworks - a $250 software package 
that fncludes wordproce$$1ng, 
spreadi^heet, and data base func- 
tions. It ts easy to use and has an 
advantage over separately purchased 
programs because It uses the same 
commands for each of the three func- 
tions. Appleyorks runs on an exten- 
ded Apple I IE (1Z8K) and needs two 
disk drives.. 



Computer sStud^it Pvofi «9 

Northwest Regional Educational 
Laboratory (NWREL) Is developing a 
student performance profile that 
Includes academic performance, atti- 
tude, and social behaviors. Each of 
these t^ree categories has a differ- 
ent f^^xh base ci.Hed from existing 
'ichuol records » For more Informa- 
tion contact;: Robert E. Blum, 
Northwest Regional Educational 
laboratory, 300 S,W. Sixth Avenue, 
Portland, Oregon 97204 (50J- 
248-6800) . 



od Article 

Keen, P.G.W. (1975) ^'Computer-Based 
Decision Aids; The Evaluation 
Probl em/ Sloan Management Review , 



The author points out that a major 
benefit of Information systems ~ 
providing better Information which 
In turn should lead to better deci- 
sions Is hard to assess with a 
tradi t1 onal cost/benef 1 t anal ysl s . 
He proposes an approach to evalua- 
tion which stresses 1 Inking the 
evaluation process with the goal- 
setting process and dealing head-on 
and up front with qualitative is- 
sues. Keen crtnslders a major prob- 
lem to be defining what a successful 
system Is, He suggests that Inter- 
ested parties — top management* 
users t desi gners — need to spend 
time In negotiating a consensus on 
"succe:s/ They also need to decide 
what the trade-offs are and agree on 
success Indicator-,. 
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CONFLReHCE ON INFCRHATION lifSllMS 
FOR SCHOOL IMPROVEMENT 



Progress Report 

A small Interdisciplinary working conference Is being pUnned 
for February 6th and 7th » 1985, un the subject of educationat inf<jr- 
matlon sy steins. 

Because of limited funds, the conference wHI be restricted to 
approximately 30 interested participants who wilt spend two days 
focusing on this topic in a variety of presentation and discussion 
formats,* 

It Is anticipated that conference topics will be aisse(i<>led iflto 
three clusters. 

In the first cluster will be presentations <ifi^^^r^q with the 
State of the Technology. Anong the topics that may be Included here 
are: the various hardware systems and their costs and benefits for 
schools; the^Hkely state of'the industry five years down the road as 
the shakedown takes place; software currently available for 



Individuals who have already agreed to participate: Bill Cooley 
(IROC), Bob BliiB (HWRL), Pat Millazo. Ed Brown, and Carol Thomas 
(SWRL). Leigh Burstefn and Ken Sirotnik (CSEl, Jean King (Tul ant? 
University). Mike Patton (University of Minnesota), Terry Canni tg'* 
(Pepperdine University). Others who will be contacted; Lynn Marku; 
(UCU/OSM). Rob klfng (UC Irvine/CPP). Ian HUroff (USC/GSK). Bill 
Ridley (MIE). .^ete Idstein (Christina School District. Newark, Of l. 
Walter Hathaway (Portland (OR) Public Schools). Hick OessauU 
(Sheboygan (WI) School District), Steve Araubel (Montgomery Co. 
School Oistritt), as well as selected superintendents, principals. 
Khool' board member^^ and representatives from 1ndu«itry, thip Statp 
Depart of Education, technical ass < stance centers , ^r^ri Mnor>^\ ■ 
f*(iucat 
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adufntstrators «nd It can be used for Instructional support 
systeett; factors to consider In buying or developing software. 

In the second cluster will be presentation:, dealing with the 
Sute of fiesearch and Oevelopieent. Anong the topics to be Included 
are: lessons to be learned fro« awnageiient Information systents 
literature; the social end organizational lapacts of computers; the 
changing roles of administrators; teacher reactions to Infonaatlon 
systems; the developiMnt of a student performance profile; the 
devfdopMnt of a school surve^y. 

In the third cluster will be presentatlens dealing with the 
State of the Practice. fmw% the topics to be Included here are: 
what high schools are doing to computerize Instructional management; 
;ase studies of districts who are Installing and using Information 
systems focusing on problems and solutions. 

Immediate y after each of the cluster presentations^ partici- 
pants will discuss what they have heard In relation to Its Implica- 
tions for policy, for research and for practice. By the end of the 
conference, participants will have produced a written list of sugges-^ 
t Ions/ Issues/concerns for each area. 'R'.roughout the presentations 
and discussions* participants wlU be pressed to address the issue 
underlying -he conference: How will Information systems Improve 
ichooU and student learning? 

Ut expect that the conference proceedings - papers, discussion 
coaments. and suggestions for ocllcy, research and practice - will be 
Qu(ck1v assembled and published. The audience for such a book will 

137 
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be administrators Interesttd In developing and Installing Infomwtlon 
systiHw. along with the technical assistance providers and lay per- 
sons who will be asked to establish, maintain, and use the process. 
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PAPER PRESENTED AT ERS/EMet CONFERENCE' 



"Concerns About Moving Ahead on Instructional Information Systems 

In School Districts" 

Richard C. Williams and Adrlanne Bank 



*Present«d as part of a CSE-organlzed symposium. The Role of Information 
Systems In Educational Excellence: Central? Ufgfui? irrelevant? 



Dangerous? , ER^/Ewet, san Francisco, October 1984, 
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CONCERNS ABOUT MOVING AHEAD ON INSTRUCTIONAL INFORMATION SYSTEMS 

IN SCHOOL DISTRICTS* 



Richard C. Williams and Adrlanne Bank 

./ 

A School District Meeting 

Recently we attended a meet+fig In a local school district. We were 
there to make arrangements whereby we could work with the district In 
helping them Implement an Instructional Information' system that would bB 
placed In local school sites. The system was based on an Innovative soft- 
ware package that had been developed by two district teachers, t^mn^ 
tially, the software allowed each participating elementary school, wslng an 
Apple IIC computer, to record electronically Individual student ciata such 
as: attendance, NRT scores and growth over specified periods of t1rae» 
English language proficiency, basic skn|ls proficiency test mults, 
schools and classrooms In which the stucjent had been enrolU^^ 

With these data available on a mlcrd-computer In the principal's . 
office, It would be possible to quickly alnd easily manlpuUte Instruction- 
ally relevant data In order to gain Insights Into pupil characteristics 
whether Individually, as parts* of a selected cohort, or of the entire 
school population. Arwed with that Information the school principal or 



staff could begin to raise interesting questions about the effectiveness of 



the school's Instructional program. 

In attendance at the meeting was t^e assistant superintendent for 
Instruction, the developers of the software package, the district's central 
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computer supervisor and the three principals who were going to participate 
in this program. We had talked to the principals prior to the meeting and 
they were interested' in the proposed program but were unclear about how * 
exactly It was going to operate and the role that they and we would have in 
• it. 

Scon after the meeting started, an intense and hea|:ed discussion 
emerged between the local software developers and the central mainframe 
computer supervisor. The gist of the discussion wa% whether it was better 
to keep these data Isra micro-computer at the local school site or to input 
the data once on the district's mainframe computer and then transfer it to 
the local site. The discussion »noved to the relative merits and possiblli- 
ties of single inputting. Uploading and downloading, and soft-discs and 
, hard-discs began to dominate the conversation between the computer ex- 

perts. The discussion eventually turned briefly to what was possible to do 
with computers and what was Important to do with computer-based information 
systems. - 

We watched the principals' body language during the discussion's 
shifting cirectlons. When the discussion began, they were sitting forward, 
talking and exhibiting other Indicators of Interest. However, as the level 
of technicality rose and the complexity of the proposed system became appa- 
rent, we could see their shoulders slump and their Interest in and under- 
standing of the proJe<?fe wane. When the discussion finally returned to * 
their original area of interest - what could the system do for them - their 
energy -level was very low. 

We share this vignette because we feel it typifies what is happening 
in meetings at many school districts around the country. As district 

C 
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administrators acquire more and more' computer softr* and hardware and com- 
puter knowledge, they are realizing that much Instructionatly relevant in- 
formation which had previously been stored In non-electronic and non- inte- 
grated files could be Integrated into a central or decentralized computer- 
based information system. They realize that data presently found In the . 
principal's office apd the teachers^' and counselors' files could be brought 
together In one data file. and, once that is accomplished,, information could 
be "skimmed off" by decision-makers at the classroom, school site, district 
administration and school board level. 

But starting with this good idea, there are a myriad of problems and 
consi (derations that seem tangled together. An understanding of the avail- 
ability, capacity and costs ^f hardware and software are needed before one 
can begin to guess at how the system can Improve on current operations. 
Vfttat can be done, what can't be done, what can be done only with Ve«t dif- 
ficulty, how much all Jhis costs, who wants or doesn't want the system, who 
can design and maintain the system, must all be considered at the same 
time. When the subject gets opened up, what once seemed like a wonderful 
dream often gets transformed into a continuing nightmare in which people 
see themselves being hopelessly trapped. 

Many districts are moving, some tentatively and cautiously, some with 
a good deal of excitement. Into the information age. They are beginning to 
see that there are opportunities to use outside experts and their own 
Internal "idea champions" build a socio-technlcal system that might improve 
what we know about teaching and learning. We refer to such a system as an 
Instructional information system to distinguish it from other computerized 
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school managji«ent systems which support administrative functions such as 
payroll, .inventory and attendance. ' 
Expressed Concerns 

Based on our experience in this and other districts we would like to 
list some of th^ frustrations and fears we've watched emerge during early 

* 

discussions of Instructional Information systems. We'd then like to 
summarize a very enlightening article by our colleague, M, l^nn Markus {now 
at UCLA's Graduate School of Manageijient) , called "Power, Politics and MIS 
Implementation"* which provides a framework with1i:» which to view these 
frustrations and fears. ^ 

Concern #1. Mho is goin^ to do all the work Involved^ The conversa- 
tion at the district meeting was, early on, dominated by the person respon- 
sible for data processing who was knowledgeable about what was currently In 
the files of the central office, and by, the programmers who envisaged a 
flexible school board system that could be easily accessed by teachers and 
principals. The principals, at though interested In having easy access to a 
newly created system, worrfed about the burden of data Input on clerks who, 

they felt, already had too much to do. Only when the programmers described 

■ # 

hove the system would relieve the clerks of most of their attendance 
recording responsibilities, leaving them free to do other data Inputting, 
did the principals want to move on to other aspects of the system. 

Concern #2. How will thU iYStem chan^ what we do now? Teachers and 
administrators are busy p*»ople with the immediate press of students in a 
classroom and school building. This reality weighs heavily upon teachers 

*ConiiimHc«t1i)n» of the ACH . Junt 1983, Mo. 6 , 430-444. 
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and administrators and causes them to value practical and Immediate, solu- 
tions to their problems. They are understandably Impatient with theoreti- 
cal solutions that have not been practically demonstrated. Additionally, 
they are accustomed to dealing with Individuals and problems as unique 
instances. They find it difficult, Irrelevant, or misleading to try to 
apply aggregated data .to Individual cases. 

One result of this preoccupation with the "here and now" and with the 
Individual is that teachers and administrators rely on solutions that are 
known bec^ause they have been derived from their own or others' experience, 
and are therefore familiar, comfortable and trustworthy - which are the 
most appealing attributes of working knowledge (Kennedy, 1982). When 
computer consultants or advocates begin talking about "hard and soft discs, 
.interfaces, uploading,," etc., teachers and administrators begin to move 
into the "unknown, uncomfortable and suspect." It is understandable that 
the principals in our meeting evidenced "shoulder slump^" What was more or 
less working for them now seemed to be on the way to being replaced by 
something which might not work as well for them. 

Concern #3. Wha t .if we start something we don't know how to finish? 
Information systems are usually designed with a particular goal in mind, 
namely, to provide appropriate and acci^rate Information to decision makers 
in a timely fashion. While this goal is laudable, and can be often 
attained, principals and teachers can Imagine unanticipated consequences ' 
that occur on the way to and during the implementation and operation of 
information systems. For example* what happens if unauthorized people use 
the system and enter Inaccurate data? What if they change records and no 
one knows about It? Or, what about existing data being used by authorized 
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persons but In unauthorized or unexpected ways. For example, data on pupil 
achievement In each teacher* s classroom might be collected in the spirit of 
helping the teacher Identify students with Teaming dlfftculttes. but there 
Is no reason to assume that the same data cannot also be used to compare 
the rate of pupil progress In different teacher classrooms at the same 
grade level. What advance assurances can be given that this would never 
happen? Once we start collecting Information that everyone can know, 
things! might begin to change. As Argyrts (1979) says: Information systems 
can substantially alter the power structures In an organization. Knowledge 
Is, after all, power. If access to^llmlted Information gives sonie people 
In^some units power over others, then the more widely the Information Is 
shared, the rrtore organizational power Is redistributed. As people's Infor- 
mal motfes of making decisions become more explicit, they may feel them- 
selves hemmed In or scrutinized. Or, they jwy real Ize^^^^^^^^ 
to make changes In their own styles of thought and behavior. 

Concern #4. How are all the reports we get going to help us do Qur 
jobs better? If data from an Instructional Information system are going to 
be used then they must be presented to Individuals In a format that he or 
she easily understands. If reports do not come on time, are complex, cum- 
bersome, Irrelevant, Impractical, then It 1$ unlikely that the Information 
will be used. The classic example of such unused reports Is when a class- 
room teacher Is given a many-page computer print-out of the standardized 
test scores of children In last year's class. Busy teachers do not have 
the time to sift the data for relevance to this year's class to derive 
Implications for tomorrow's classroom act^vltlp^. However, although many 
may not use these reports, a residue from their non-use remains; teachers 

1 45 



may fear being seen as not smart enough to understand high-status data, or 
they may worry that the reports contain something they should know about 
but do not. 

It has become self -evident that people In different roles - In this 
case, classroom teachers, principals, board members - have different Infor- 
mation needs In terms of the types of content they need, the format c.nd the 
frequency with which they get reports. The same data might have to be 
arrayed and analyzed In quite different ways for people In each of these 
different roles. 

As Individuals, adults differ from one another In terms of their cog- 
nitive styles. It appears that some people relate more readily to graphic 
presentations of data while other pay more attention to written text. Some 
Individuals want to be presented with patterns over time. Others prefer to 
see exceptions fl agged. SpM pePPTe wjnt see "isverything;" others want 
"only what I need ..." ^ 
Analysis of Concerns 

The foregoing concerns were expressed In one form or another at the 
meeting we att^d^d. The concerns, we believe, are not unique to the par- 
ticular district we were In; not are these concerns unique to educators. 
The management Information system (MIS) literature sometimes discusses such 
concerns under the term "resistance." Resistance Is provisionally defined 
by Markus as "behavior Intended to prevent the Implementation or use of a 
system or to prevent system designers from achieving their objectives" 
(p. 433), 

Markijs analyzes three^itheorles of resistance and her analysis is 
addressed to the system designer or to the system manager who may take 
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different courses of action In dealing with concerns such as those already 
listed depending on which theory they subscribe to. 

The three theories are described as "people <:,4tenn1ned," "system 
determined," and "interaction determined," 

InHhe "people determined" theories, the causes of resistance are 
perceived to He within individuals' cognitive styles, or personality 
traits. The assumption is that resistance is an attribute of the user and 
1$ undesirable because the purpose of the system being introduced is con- 
sistent with rational management theories and because the organiiational 
goals which the system Is serving should be ^shared by all participants. 

In the "system determin^id" theories, the courses of resistance are 
perceived to He within the proposed system Itself, e.g., lack of user- 
friendliness, inadequate technical design or implementdtion. The assump- 
tion here is that resistance is a realistic assessment of system 
Inadequacies. 

In the "interaction" theories the causes of resistance are perceived 
to lie within the interaction of the system and the context of use. Resis 
tance is a product of the setting, the users, and the designers, r is 
neither desirable nor undesirable. "Resistance is not a problem to be 
solved so that a system can be installed as Intended; it is a^iseful clue 
to what went wrong and how the situation can be righted" (p, 441). 

System designers or system managers subscribing to the people- 
determined theory of resistance will try to overcome user resistance by 
education and training of uters; or by coercion, sanctions, or per^iuaslon. 
They will develop strategies to obtain user involvement so as to obtain 
their commitment. 
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System designers or. system managers subscribing to the systeJi?*- 
deteriRinedl theory of resistance will try to educate designers so that they 
develop better technology or modify their procedures and packages to con- 
form to organizational procedures.. They will develop strategies to get 
user Involvement so as to get better system design. 

System designers and system managers who subscribe to tKe Interaction 
theory will try to: fix organizational problems before Introducing sys- 
tems, restructure l.icentlves for ust.s, restructure-ieleitlonshlps between 
users and designers. Accor(^1ng to this theory, usepl participation Is not 
always necessary or appropriate^. 
Dealing with Concerns 

Although we were observers at the district's meeting described at%he 
beginning of this paper, we anticipated that we would be able to provide 
some assistance la easing the Introduction of an Information system Into 
.the schools. We were familiar with the "rules of thumb" suggested by the 
MIS literature and by our own previous case studies of school districts to 
overcome Initial resistance: get tpp management support, get user Involve- 
ment, inake the system userr friendly; design a system that has Immediately 
perceived benefits; start small. 

"»«» 

rhese prescriptions are derived from either the "people" or the 
"system" theories of resistance. While we believe that' these are still 
valuable tips to keep In mind, we will use additional insights suggested by 
the "Interaction" theory In an attempt to facilitate the design and Imple- 
mentation of an Instructional Information system that will work. These 
might Include 1) powv* r: discovering who wins and who loses by the Intro- 
duction of an information system into the schools; 2) culture : exploring 

lis 



141 



no*i information systems may change the divUlon of labor and organizational 
relationships and norms; 3) Image : Investigating how the proposed Informa- 
tion systeni may affect and be affected by the image that people have of 
themselves, their school. the1r*d1 strict, the commi'nity. 
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